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ABSTRACT 


Asymmetric threats pose increasing challenges to combatant commanders in Stability, Security, 
Transition, and Reconstruction (SSTR) Operations. Threats once confined to littoral waters 
now extend out to the green water theater. Many NATO countries operate their post Falkland 
war built frigates in these missions, in which lightly armed small, agile and fast craft are often 
encountered. 

This study uses freely available real-world data to build a simulation using an agent-based 
modeling platform called MANA. The simulation is exercised over a broad range of factor 
settings that are determined by an efficient experimental design. Using linear regression and 
partition trees, a robust analysis is performed on the resulting dataset to create statistical models. 

Conclusions gained through these models suggest that a swarm of small craft, armed with hand¬ 
held weapons, could attack and achieve a mission kill on a typical NATO FFH operating in a 
SSTR mission. It further implies that the FFH’s mission survival is dependent on the sophisti¬ 
cation of installed weapons, and that a mix of advanced, automated weapons is best suited for 
close-in defense against multiple small seaborne attackers. Therefore, it will benefit mission 
survivability to improve or replace existing body-aimed weapons of frigates serving on SSTR 
missions. 
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CHAPTER 1: 
Introduction 


According to the U.S. DoD Maritime Strategy (Carmel, Hughes, Pendley, Rubel, and 
Till, 2008) traditional and espeeially asymmetrie threats eontinue to pose inereasing ehallenges 
to any eombatant commander in a notional Stability, Seeurity, Transition, and Reconstruetion 
(SSTR) Operation. Limited threats that were onee eonfined to brown and littoral waters now 
extend out to the green water theater (U.S. Department of Defense, 2007). As building new 
frigates to speeifieally counter these threats is a very expensive and time eonsuming effort (De¬ 
fense Industry Daily, 2008), it is neeessary to investigate how well a typieal North Atlantie 
Treaty Organization (NATO) operated Fast Frigate with Helicopter (FFH),^ built between 1975 
and 1990 and operating as a lone unit, would perform in a SSTR mission when attacked by 
small, agile fast eraft armed with hand-held weapons. 

A hypothetical defense against different sized swarms of small vessels at different speeds 
in differing environmental eonditions is simulated. Different frigate defenses are modeled using 
a range of old and new armaments. The intent is to provide answers to the questions as enumer¬ 
ated in Chapter 1.1. 



Figure 1.1: Koln Class Frigate Koln IV, F220 (From U.S. Defense Imagery, 1985). 


'in this study. Fast Frigates with Helicopter (FFH) and Fast Frigate, Guided Missile, with Helicopter (FFGH) 
are used as synonyms. The Guided missile does not influence proposed research. Please see Chapter 2 for further 
explanations. 
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1.1 Research Questions 

1. Can a swarm of four to eight small agile boats armed with handheld weapons, attack 
and achieve a mission kill on a typical NATO FFH operating in a SSTR mission? 

2. Is the FFH’s mission survival dependent on the sophistication of installed weapons? 

3. What type of weapon or weapons package is best suited for close-in defense against four 
to eight small seaborne attackers? 

4. Would it benefit mission survivability to improve or replace existing weapons of frigates 
performing SSTR missions? 

All testing for answering the research questions rests on the premise that the new or 
updated capabilities can at least theoretically be equipped on the average commissioned frigate 
while not, or only minimally, altering the ship’s hull or its hull structure respectively. 

1.2 Background and Motivation 

The whole of the Mediterranean Sea, as well as the merchant routes between the Mediter¬ 
ranean Sea and the Persian Gulf, are strategic and economic assets for many NATO countries 
(Yost and Masala, 2005). Preserving the freedom of the seas, including the free passage of trade 
routes, and denying access to these resources for pirates and other asymmetrical threats is of 
vital importance (U.S. Department of Defense, 2008). NATO doctrine states that piracy and ter¬ 
rorism are partially interdependent (North Atlantic Treaty Organization, 2009). Therefore, task 
forces are stationed in both areas to demonstrate vigilance,^ and at the same time control for 
those threats. Since the attack on the USS Cole on October 10, 2000, previously unrecognized 
or underestimated threats emerged for navy vessels engaged in SSTR operations. 

Vessels in brown waters recognize the potential threat of small craft armed with hand¬ 
held weapons or stacked with explosives, attacking from within the cover offered by other 
neutral vessels and using the element of surprise (News Navy.mil, 2009). 


^Both Operations Active Endeavor and Enduring Ereedom evolved out of NATO’s immediate response to the 
terrorist attacks against the United States of September 11, 2001. The Operation’s goals are to monitor shipping 
and provide escorts to non-military vessels and to help detect, deter and protect against terrorist activity (NATO 
Operations, 2009). 
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In green water engagements, these threats ean take the form of small, fast, relative lightly 
armed vessels in possibly large numbers, and disguising themselves as legitimate vessels prior 
to attaek. Sueh attacks might be performed by terrorist organizations, as with the USS Cole, 
or as part of the increase in pirate activities currently observed in the Gulf of Aden and other 
comparatively unstable areas around the globe. The first in a continuing series of examples of 
this kind of activity is the sinking of a suspected pirate vessel by the USS Gonzalez on March 
18, 2006, near Somalia territorial waters (Besheer, 2006). 



Figure 1.2: Pirates off Somaiia in a skiff, whiie ciosing in on a cruise iiner in 2005 (From Associated Press, 2008). 

The vicinity of the Gulf of Aden (GoA), while geographically compact, is crowded with 
commercial transport, recreational cruise ships, and a multitude of fishing vessels. Many of the 
local fishing vessels are in very poor condition. The same is true for the vessel’s crews, whose 
sole survival is often dependent on whether they catch enough fish to eat and sell. The GoA 
also features calm seas, allowing for extensive use of small and fast vessels. Since this region 
also borders an unstable area, as well as a state hostile towards NATO, it is a prime candidate 
for an attack to occur (Figure 1.3) (Wright, 2007). 

Commercial transport and recreational cruise vessels are easily identified by radar cross- 
section, speed and the mandated automatic identification System (AIS), but the local fisheries 
are not industrialized as in many western countries. The typical fishing vessel is built in the 
style of a dhow, made of wood, possibly equipped with a dated VHF transmitter, and more of¬ 
ten with a Thuraya satellite phone than a radar system. 
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Some dhows are aeeompanied by small, agile and quite fast boats used to herd fish. 
These boats will further on be referred to as “small, agile fast craft” (SAFC). This name is 
chosen to differentiate these civilian craft from fast inshore attack craft, which in U.S. Navy 
terminology are specifically procured as military attack vessels, chosen with the specific intent 
to attack larger military^ vessels in large numbers (Navy.mil, 2007). To identify these dhows 
and the accompanying SAFC, it is necessary to approach them and visibly read off the ship’s 
name, as the physical differences between individual dhows are minute. Some of the speedboats 
accompanying the dhows are only detected by the wake they produce when at high speeds. 

Many NATO countries participate in patrol missions, each sending different types of 
frigates, most of which were built between 1975 and 1990. Most of these frigates were de¬ 
signed for convoy protection and submarine hunting. As such, they have good surface radars, 
anti-ship missiles and anti-submarine capabilities but not a lot of close range defenses against 
small craft. They are either not or only minimally armored. 
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Attempted Attack ^ = 


Suspicious vessel 


= Actual Attack ^ = 


Figure 1.3: International Maritime Bureau Piracy Map for 2008 (From International Chamber of Commerce, 2008). 
^And also civilian vessels, for example during the tanker Wars with Iran (Navias and Hooton, 1996). 
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When encountering SAFC in the GoA, most NATO Navy’s standard operational proce¬ 
dure is to advise the SAFC to steer clear of the frigate. Usually the SAFC will not react as they 
have no means of communication'* and will continue along their path towards the frigate. It is 
then customary to avoid these vessels while still trying to assess the crews and determine their 
intent. As long as the weather permits, many of those vessels are faster, some considerably, than 
a frigate. It is nearly impossible to evade these craft even when they are not aiming to intercept 
the frigate. Additionally, it is hard to evade the vessels and simultaneously identify the crew 
or possible weapons. An adequate visual inspection requires approaching the vessels until they 
are at most half a mile away. At this distance, assuming the small craft have an excess speed 
of 10 knots in relation to their intended target, they will reach the frigate within at most three 
minutes. Given the typical size of a NATO frigate, this is not a lot of time, and certainly not 
enough to begin issuing combat stations and expect a stringent defense. 

The motivation of the SAFC to attack a NATO frigate is thought to be the humiliation 
and the subsequent retreat of that enemy. This might either be motivated by some political ide¬ 
ology, or maybe the attacker’s business model of generating ransoms for the release of captured 
vessels is currently impeded by the frigate’s presence in the area. The actual motivation is only 
of interest insofar as the attack is not thought to be a suicide commando, but rather someone 
who is interested in surviving the encounter. 

The objective of SAFC during the attack then is maximum media coverage and the 
disgrace of the stronger and technologically advanced enemy. As the attack occurs at sea, to 
achieve and amplify the media effecC the attack must either be recorded or, for even better 
results, the attacked ship needs to be able to return to port, with visible damage, exploiting full 
media coverage. In this scenario, the frigate will therefore have lost a simulated encounter once 
a mission kill is determined, allowing us to view the frigate as a single entity without the need 
to model every and all compartments and machinery. Also, as similar in armament as most 
NATO frigates are, some quite different building techniques are employed. Using the approach 
discussed, all these different commissioned frigates can be reduced to one general design, with 
varying numbers of hits admissible until the frigate is assumed to be a mission kill (for an 
overview of former and current frigate designs, see Appendix B). 

'^This fact is based on observation in 2006 only. 

theory that relates how stories published in the media influence or amplify current trends (Bryant and 
Thompson, 2001). 
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For the SAFC, up to four attackers per craft are assumed. The total number of occu¬ 
pants per craft might be higher, but four is thought to be a minimum for this type of exercise. 
One person will steer, one will fire a handheld weapon and up to two will fire rocket propelled 
grenade launchers, like the RPG-7 or a variant thereof.^ 

Further details of interest in this study are presented in Chapter 2. 


1.3 Scope of the Thesis 

This thesis describes an abstract model of a common NATO FFH type Frigate operating 
in a SSTR scenario, with the different scenarios being simulated using Agent Based Modeling 
(ABM). The small, agile fast craft will attack the frigate in different environment conditions, in 
different numbers and with the frigate employing defense systems of different kinds and vary¬ 
ing sophistication, at the same time varying the hit probabilities of both attackers and defenders. 

Armament, skill, speed, approach vectors and number of the SAFC will be varied. The 
attackers are positioned at random on the map during initialization. The survival and the perfor¬ 
mance of the SAFC are not issues to be researched; therefore, these factors are not accounted 
for. As such, advanced tactics, as for example luring the FFH into a trap and other such forms 
of attack or special formations are not studied. 

The SAFC hit probabilities, as well as evasive maneuvers, are balanced assuming the 
best attack strategy is being employed. All engagements are assumed to occur during the day. 
To find and attack an enemy in green waters requires some sophistication. The SAFC need to 
have some general idea of the location of the frigate. This is represented in the model by them 
closing in from only two directions, but for the final approach they will need to have a visual 
reference. Even though low light and infrared equipment are getting cheaper, it is not assumed 
that the attackers have this equipment readily available. Therefore, they are denied the possibil¬ 
ity of a night attack. 

^The Rocked Propelled Grenade launcher 7 and its variants are the most commonly used and easiest to acquire 
large caliber weapon available, and given its effectiveness, also the cheapest. A black market armor piercing 
grenade for this launcher costs about $15, whereas the launcher is an investment of about $600 if buying the 
original. Copies from countries other than Russia (except for those built under license in the U.S.) are thought to 
be even cheaper (Gebre-Wold and Masson, 2002). 
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The survival of the SAFC is not of interest as a measure of effectiveness (MOE), as 
regular NATO procedures are to defend, call for help, and tend to any casualties if necessary. 
However, the frigates would usually not try to pursue and kill enemy craft once those are re¬ 
treating. 

The frigate is assumed to try maneuvering in such a way as to achieve good weapons 
coverage on their opponents while simultaneously minimizing the cross-section they expose to 
the enemy’s weapons fire. This will be achieved by varying the placement and speed of the 
frigate agents. 

Deciding whether the approaching vessels present a threat or not is one of the problems 
assessed in this model. NATO operational procedure and Rules of Engagement (ROE) usually 
assume neutrality of contacts until proven otherwise. The ROE of the frigate are assumed to 
prohibit a preemptive strike on the approaching vessels. Nonetheless, it is assumed that the 
presence of the SAEC is not a surprise to the frigate, and the crew will thus be at battle stations 
when the actual attack of the SAEC occurs. Once the small vessels are proven hostile, a full 
response using reasonable but necessary force will be administered. The frigate will also react 
once an attacker crosses within the safe zone around it, which will be varied extensively in the 
simulations. A craft inside the safe zone is assumed to be hostile and will be attacked. Eor 
modeling purposes, this distance not only represents the different ROE each country operates 
within, but also represents the use of non-lethal weapons (NEW) to determine hostile intent. 
Most of these weapons are only effective over a short distance, and studies show that they do 
not deter a determined attacker from continuing with or starting an attack (Sickinger, 2006). 

Adding to the problem when confronted with multiple closing SAEC is whether they 
all have the same intent, and may therefore be all classified as hostile. As this is a decision to 
be made by the commanding officer, both variants will be modeled. In one case, single vessels 
definitely classified as hostile will be attacked. In the other case, once one vessel is determined 
to be hostile, all nearby craft are valid targets. Due to the short range, an enemy being tar¬ 
geted by a NEW is possibly already within its admissible weapons range. The benefit of being 
able to discriminate friend and foe may therefore be range dependent, especially since some 
of the frigate’s weapon systems have a minimum effective distance inside which they are not 
employable. To account for all these conditions, the frigate will either react when shot at, when 
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the SAFC are determined hostile or when they cross a minimum safe distance. Each of these 
conditions will be set in advance to control for their respective or combined effects. 

All weapons that are modeled in each scenario are to be admissible weapons in accor¬ 
dance with ROE. ROE requirements allow a “necessary and reasonable response” to an encoun¬ 
tered threat. 

The study does not try to incorporate elements of advanced scouting, training issues or 
the effects of target selection. 

All damage control, as well as target allocation, is assumed to be perfectly coordinated. 
It is also assumed that no magazines or other ammunition are hit. Effects of catastrophic hits, 
such as ordnance explosions, aboard the frigate or SAEC are not modeled. In reality these hits 
may well result in an instant mission/unit kill. Eurther, elements of exhaustion, fatigue, or any 
other human factors affecting the behavior of the frigate or the attacking SAEC, are not an issue 
in this study. 

The frigate is not tested for armor effects, which may well be of significance in an attack 
involving small weapons fire. The incorporation of armor in an existing vessel’s design is not 
trivial and would very probably necessitate an extensive redesign of the hull, a condition ruled 
out in the problem statement. 

The use of a helicopter is studied. Most NATO shipboard helicopters are not designed 
for and are not effective in a surface interdiction and attack role. The U.S. Army, for example, is 
assigning ground interdiction and attack missions to fixed-wing aircraft because ground-attack 
helicopters have proved to be highly vulnerable to small-arms fire (Grant, 2006) in those roles. 
This does not apply to close air support (CAS) missions (Groenke, 2005). The only shipboard 
helicopter designed for ASuW missions is the recently introduced MH-60R Seahawk using the 
AGM-114N Hellfire II Missile (Eigure 1.4). Using this weapon system, the stand-off distance 
and accuracy is such that it is thought to influence the frigate’s survival. Many studies have 
shown a helicopter to be a valuable asset ((Efimba, 2003), and (Abbott, 2008)). Even if the 
helicopter has no weapon system comparable to the Hellfire System, and, therefore, is not as 
effective in the CAS role, it may draw some weapons fire away from the frigate and thus help 
the frigate survive. Shipboard helicopters in this model do not increase the defensive capability 
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of a frigate by warning of an armed enemy approaehing, as the frigate is assumed to be ready 
to defend itself whenever the weapon exehange begins. 



Figure 1.4: Lockheed Martin AGM-114N Hellfire II on display at ILA 2006. 

The study will be able to give a plausible insight to the armaments benefieial to a frigate 
that is expeeted to eneounter a small boat non-suieide attaek. The study will approaeh the ques¬ 
tion of whether eurrent frigates are adequately equipped to deal with the seenario they eurrently 
faee. It will not answer any question outside this very speeific realm, and any ehanges to the 
weapons mix that may be suggested by this study should be evaluated in the eontext of the many 
other eombat and support seenarios possible in the politieal environment we are living in today. 
Also, as this study develops a distillation model of all NATO frigates built between 1975 and 
1990, eertain features that may have an impaet on individual performanee may be lost in the 
study. Applying the results to any partieular frigate would require further study explieit to its 
partieular design. 

1.4 Methodology 

The simulation is set up within an agent based modeling (ABM) environment ealled 
the Map Aware Non-uniform Automata (MANA), version 4.04. This modeling environment 
is eapable of simulating the frigate, as well as different types of SAFC when they are pitted 
against eaeh other. The ehosen ABM provides a data farm-able interfaee eapable of being run 
in batch mode. The input data for the model are assembled using spreadsheet techniques, and 
are mainly based on sources such as Jane’s Fighting Ships and Jane’s Naval Weapons for tech¬ 
nical data (Saunders, 2008). Open, publicly available literature such as manufacturer supplied 
information, article reviews, and subject matter expert interviews are used for the determina¬ 
tion and later calculation of hit probabilities. Both the subject matter experts, as well as the 
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manufacturer supplied spec sheets are understood to provide general guidance rather than pre¬ 
cisely describing the true performance of any given system. The frigate design is abstract, and 
weapons performance varies with factors such as platform stability, weapon control, and sensor 
quality. When combined with the intention of exploring ranges of performance using robust 
experimental design techniques, these data sources are thought to be sufficient for the purpose 
of this thesis. Data on weapons for which neither an expert nor a technical data sheet can be 
found will be approximated by comparable designs and design ranges. 

Exploring ranges of input values rather than discrete values will mitigate the effect of 
uncertainties in the input data and capture some of the stochastic effects of warfare. Different 
model starting conditions will be varied using a Nearly Orthogonal Latin Hypercube (NOLH) 
design (see Chapter 3), and each variant will be run on a computing cluster with sufficient repli¬ 
cations to assess variability in the responses. Exploratory data analysis techniques will be used 
to characterize the impact of weapons packages and tactics on the EEH’s performance. 

1.5 Chapter Outline 

Chapter 2 explores the set-up of the model inputs. A distillation of current frigate and 
SAEC designs is explored, and weapon systems, as well as hit probabilities explained. Included 
is a brief discussion of the real-life settings, conditions and constraints that are implicitly as¬ 
sumed for the model. After a brief overview of the model’s capability, advantages and dis¬ 
advantages of using MANA versus other modeling environments are discussed. The relative 
merits of using time stepped versus discrete event simulations are discussed and the measure of 
effectiveness is explained. 

Chapter 3 contains the complete design of experiments. It describes how the frigate 
and SAEC real world representations are integrated within the modeling environment. It details 
what systems and capabilities are varied and by what amount. Eirst the frigate’s designs and its 
implementations are detailed, followed by the SAEC capabilities. 

Chapter 4 contains an overview and explanation of the statistical methods used and the 
analysis of the data produced by the simulation.The analysis section is divided up in a section 
analysing data with and a section analysing data without a helicopter present within the simu¬ 
lation. 
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Chapter 5 concludes this thesis, summarizing the results and offering follow-on prob¬ 


lems. 


1.6 Benefits of the Study 

The German Command for Transformation of the Armed Forces and those NATO coun¬ 
tries contributing FF(G)H Frigates to SSTR operations will benefit from this study. This study 
identifies appropriate defense options for any maritime vessel facing “swarm” threats that are 
employing short to medium range weapons. There is also potential for identifying training needs 
unique to a SSTR mission. The New Zealand Defense Force, Defense Technology Agency 
(DTA), is interested in the results of this study. The DTA is currently simulating a frigate pro¬ 
tecting a high value unit being attacked by multiple vessels employing red teaming. 
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CHAPTER 2: 
General Parameters 


The primary objective of this research is to use modeling and simulation to determine 
whether current NATO operated FFH designs commissioned between 1975 and 1990 are suited 
for defending themselves against a swarm of 4-8 SAFC. This chapter describes how the dif¬ 
ferent capabilities and characteristics of the distinct frigates, weapon systems and small craft 
were determined. It also shows how the frigate types and weapon systems fitting the selection 
criteria were distilled. Finally it describes the process of model environment selection, general 
model implementation considerations and explains the chosen Measure of Effectiveness (MoE). 



Figure 2.1: FFH 56 SIMPSON, an Oliver-Hazard Perry Class Frigate. 


2.1 Frigate Setup 

Many NATO countries designed and commissioned new frigates between 1975 and 1990 
(Figure 2.1). Findings made during the Falkland War in particular convinced some countries to 
remodel their existing fleet. In this subsection, all FFH designs as built between 1975 and 1990 
and still in service of NATO countries at the time of writing are examined and distilled into a 
single frigate model to be used in the scenarios. The same is done for weapon systems capable 
of targeting SAFC. Finally, given SAFC as target for the different weapon distillations, to-hit 
probabilities are determined. 
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Some weapon systems under consideration for development are not studied in this re¬ 
search. These include electromagnetic gun systems, otherwise known as “Gauss cannons” or 
“Railguns,” as well as LASER based defense systems, such as LCIW.^ Both systems are un¬ 
dergoing field tests, but both systems are very hard to integrate in one of the existing frigate 
designs, as they follow a different design paradigm — the all electric ship. 

Following the 2002 intervention in Iraq, some new active defense systems have been re¬ 
leased to the market. These systems may have a serious impact on RPG effectiveness, which, at 
least at the moment, seems to be the SAFC’s most daunting weapon. Due to the short distance 
for engagement, in the current development stage these systems rely on an armored carrier. 
The carrier needs to be armored to protect itself from both the interceptor’s and intercepted 
grenade’s explosion and fragments. A frigate does not provide the kind of armor protection 
needed to employ this system. Additionally, these RPG defense systems are built to defend a 
single small land based vehicle, such as a Main Battle Tank or Armored Personnel Carrier, and 
not a large frigate (Defense Update, 2006).* 


2.1.1 Distillation of Different NATO FFH Types 

According to Jane’s Fighting Ship, the following NATO FFH classes were commis¬ 
sioned between 1975 and 1990 and are still in operation by NATO countries (in alphabetical 
order): 

• Bremen class (Type 122), Germany 

• Broadsword class. Batch II (Type 22), UK 

• Broadsword class. Batch III (Type 22), UK 

• Duke class (Type 23), UK 

^Laser Close In Weapon System. They will not be considered in this study as they 1) are not yet and for a few 
years will not be operational, and 2) require far higher amounts of electricity to operate than is usually available 
on a vintage design. 

^Those systems use a fixed radar sensor, mounted on the protected platform, to detect potential threats, measure 
the distance and trajectory, and thus providing the fire control system with data for calculation of engagement 
plans. When a threat is identified, normally the muzzle flash in the case of an RPG, an explosive projectile or 
small missile interceptor is launched toward it. The interceptor either tries to hit the incoming projectile, or tries 
to destroy it using a fragmentation blast. Both the missile and the fragmentation blast may damage an unarmored 
carrier system in the process of defending against the incoming projectile. Compare (Defense Update, 2008) and 
(Defense Update, 2007). 
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• Elli class, Greece^ 

• Karel Doorman class, Belgium (ex Netherlands) 

• Kortenaer class, Greece (ex Netherlands) 

• Lupo class, Italy 

• Maestrale class, Italy 

• Nils Juel class, Denmark^® 

• Oliver Hazard Perry class, USA/Turkey/Spain 

• Vasco da Gama class, Portugal 

• Yavuz class, Turkey 

Not considered is the Spanish Baleares class (Figure 2.2), based on the US Knox class 
design, as the last vessel is being retired at time of writing. Also not considered is the Dan¬ 
ish Thetis class, as it is classified as an Ocean Patrol Frigate, and not comparable to the Fast 
Frigate Helicopter (FFH). The subset of the FFH, the FFGH^^ design, is considered in this study. 

For frigates considered, minimum and maximum design parameters that are thought to 
be important are extracted. This is done in order to cover the complete range of capabilities for 
each frigate within the selection criteria. After this process, the following generic frigate set-up 
is determined: 

• Maximum Speed using main propulsion: 21 - 34 kt 

• Maximum Speed using economic propulsion: 14 - 26 kt 

• Tonnage: 1320-4900 

• Fength: 84 - 149 meter 

• Width: 10.3 - 16 meter 

modified Kortenaer class, designed when this class was sold to Greece. 

^°In international comparison, mainly due to its small size and light weight, this type should be rated as a 
corvette. As it is registered as a frigate, it is included in this study. 

^'Fast Frigate Guided missile Helicopter. 
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Figure 2.2: FFH 71 BALEARES, a Spanish Knox type variation. Last of the ciass was decommissioned in December 
2008. 

• Main Gun(s): 1 x 76 mm -lx 127mm (2 x 76 mm) 

• Auxiliary Guns: none - 4 x 30mm part automatic 

• Small Guns: 2 x 7.62 mm MG’s - 4 x 12.7 mm MG’s 

• CIWS^^ Gun: none - 30 mm full automatic 

• CIWS Missile: none -2x21 rounds automatic 

• Surface to Air Missiles: none able to target small craft 

• Surface to Surface Missiles: none able to target small craft 

• Number of radar fire controls: 2-5 independent 

• number of optronic fire controls: none - 2 

• Helicopter firing AGM-114N Hellfire: none - 1 

With this set-up, the next step is to determine the actual weapon types used within the model. 

^^Close In Weapon System, a last line of defense against low flying missiles. In recent upgrades. Helicopters, 
Small Boats and larger Vessels may be engaged. This is also true for the recent CIWS Missile systems. 

^^See the appendix for a full overview of individual frigate’s capabilities. 
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2.1.2 Weapons Types Considered in the Model Set-Up 


Even though there are thirteen different frigate designs determined to fit the seleetion 
eriteria, it would be interesting to understand what weapons in what eonfiguration would aetu- 
ally make a differenee in a battle. Nonetheless, the aseertained design still needs to be realistie. 
In this subseetion, all weapons eonsidered for the model set-up will be briefly deseribed. 

Missiles 

Most missile systems as installed aboard a FFH are employable against larger size tar¬ 
gets only, as the missile will either not loek onto a target the size of a SAFC, or the prepro¬ 
grammed detonation area is not at the height and depth of a SAFC.''^ Thus, anti-ship eruise 
missiles and anti-air missiles are not eonsidered and not modeled for these runs. 

Main Guns 

A “Main Gun” is an artillery pieee that is not clearly defined in size or purpose. Most of 
the frigate types investigated employ a 76 mm automatic cannon as the main artillery armament. 
The main gun on a frigate from the selected era is typically installed for air defense purposes, 
a feature that may be easily deduced by the placement of the 76mm gun aboard Oliver-Hazard 
Perry type frigates. This design places the main gun in the middle of the deck house, in between 
the main mast and the exhaust structure (see Figure 2.1 in combination with Figure 2.3). 



Figure 2.3: USS Curtis' OTO Meiara 76mm (Mk75) gun engaging. 

'"^Modern Missiles will try and detonate three to five meters within an enemies’ hull, and penetrate about one 
to two meters above the water line. Therefore, most common missiles will pass any SAFC overhead and will not 
detonate, even if they had build-in multiple attack software (Gomar et al., 1996). 
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Item 

min 

max 

measure 

ammunition 

5 

90 

ready Rounds 

gun train 

15 

40 

degrees/second 

target acquisition 

10 

40 

seconds 

inter-firing time 

0.5 

3 

seconds 

firing arc 

210 

270 

degrees 

engagement distance 

150-500 

60000 

yards 


Table 2.1: Main Gun System characteristics 


Even though this paradigm has changed and artillery is no longer specifically employed 
for air defense purposes, the guns of the selected era are specialized for AAW duty. Some coun¬ 
tries have upgraded the main gun’s soft- and hardware to enable firing on small vessels.With 
legacy software, a hit with the main gun on SAFC type vessels is very hard to achieve, even in 
the best of circumstances.'^ The most prominent and widely used exponent of this class is the 
OTO Melara'^ 76 mm naval gun, also listed as the US MK 75 (Friedman, 2008). 

On the other hand, more recent gun systems like the Bofors 57mm MK III (Figure 2.4), 
use an advanced ASuW mode employing advanced shells and digital gun control. This allows 
for different projectile behaviors, for example to detonate the projectiles directly above a target, 
dramatically increasing P(hit) especially for small targets. It is also worth mentioning that guns 
of higher caliber and short inter-firing times are often subject to jamming and/or misfires. Mod¬ 
em gun systems tend to have fewer of these reliability issues.'* A direct or adjacent hit of any 
Main Gun projectile will achieve an instantaneous kill of a SAFC (Table 2.1). 


^^Compare the most recent upgrade programs for Type 122 and Type 123 frigates of Germany, or the upgrade 
to hard- and software of US 5” guns now being able to fire a so-called shotgun shell, MK182 KE-ET (Steelman, 
2005). 

^^Historically, these guns had two modes of operation, AAW and an ASuW . The ASuW mode was a backup 
and last resort to attack a larger hostile vessel. As such, the projectile is shot in a way to penetrate the target about 
2 to 4 meters above the waterline, and detonate 3 to 5 meters after impact (Null and Wallace, 2003). 

^^After a merger this gun is also known as OTOBreda. 

^^Compare the problems the British Navy experienced during the Ealkland War with the advanced MK8 gun 
system (Eischer, 2006). 
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Auxiliary Guns 

Humans, especially when not constantly trained, are poor marksmen shooting medium- 
caliber auxiliary gun systems.'^ The guns themselves are usually quite accurate. Movement of 
the gunner, the gimbals, the ship and the target hamper good performance. Additionally, gun 
training is quite expensive, needs a huge free range, moving targets to produce effective training 
and is therefore time consuming. Whatever the reason, auxiliary gun training nowadays is not 
a common occurrence. In recent years, in light of the poor performance of human tracking, 
automatically tracking gun mounts have been introduced (Watkins, 2005). The human operator 
only needs to point the gun at the intended target, and the gun will acquire, follow and typically 
hit the intended target with great accuracy. 



Figure 2.4: An air burst of a 57mm Bofors MK III projectile above its intended target. 

Recent auxiliary gun designs further increase both lethality and kill probability by use 
of advanced projectiles. New designs either use the High Explosive (HE) timed round (Eig- 
ure 2.4), the Erangible Armor Piercing Discarding Sabot^° (EAPDS), or the Armour-Piercing 
Pin-Stabilized Discarding Sabot (APPSDS) (Pigure 2.6).^' 


*®As an example, during World War II land based air raids an Anti-Aircraft Battery on average used between 
a few thousand and 18500 rounds of ammunition for hitting and destroying just one aircraft, at day and night 
respectively. These hit rates were for stationary guns — additionally the speed of aircraft was comparatively slow 
during that time period (Baldwin, 1999). 

^‘’This penetrator consists of a brittle tungsten alloy, designed to fragment after impacting a hard or soft target. 
The “round disintegrates into fragments, which then progressively break up into a cascade of ever-smaller particles. 
The result is a high-energy cloud of fragments” (Gander and Cutshaw, 2007). 

^'The Bofors 57mm MKIII gun employs this most recent development. Using an on-mount muzzle velocity 
radar, the rounds are programmed while traversing the muzzle to burst in air. Using this method, the rounds 
detonate within a 2-5 meter circle. This is especially useful for small boat defense (Watkins, 2005). 
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Item 

min 

max 

measure 

ammunition 

6 

100 

bursts 

gun train 

10 

45 

degrees/second 

target acquisition 

2 

25 

seconds 

inter-firing time 

1 

4 

seconds 

firing arc 

120 

210 

degrees 

engagement distance 

50 - 200 

4500 

yards 


Table 2.2: Auxiliary Gun characteristics 


It may be important to point out that common frigate designs especially from the late 
seventies did not sport any auxiliary gun systems, since in the scenarios discussed in those times 
they were deemed incapable of aiding in the protection of convoy operations, ASW operations 
or the defense of the frigate itself (Table 2 . 2 )}^ 

Close In Weapon System (CIWS) 

At the time of writing, two types of CIW systems are being deployed, missile based and 
gun based systems. 



Figure 2.5: RAM launcher prior to HAS upgrade aboard a frigate. 

^^The real threats were seen as guided missiles and missile equipped aircraft shooting from very long distances, 
therefore close in defense was secondary. 


20 














Item 

min 

max 

measure 

ammunition 

8 

14 

bursts 

gun train 

30 

60 

degrees/second 

target acquisition 

5 

15 

seconds 

inter-firing time 

3 

20 

seconds 

firing arc 

180 

360 

degrees 

engagement distance 

150 - 300 

4000 

yards 


Table 2.3: CIWS (Gun-based) characteristics 


Gun based CIW systems fire a stream of projectiles to intercept an approaching enemy 
missile, with between 90 and 200-1- projectiles per burst. Current gun designs have around 10- 
15 bursts until they need reloading. If possible, a complete at-sea reload takes anywhere from 
15 to 60 minutes,^^ depending on how the gun was integrated and how much ammo is loaded. 
Both the GOALKEEPER and the Phalanx CIWS models have inherent tracking and intercep¬ 
tion capabilities for air targets. Recent versions also have a high hit probability for seaborne 
vessels of small to large sizes (Table 2.3) (Martin, 2000). 

Missile based CIW systems fire one or more short range missiles at high speed at their 
intended target. They intend to either kill through kinetic force or through detonating a small 
charge and spraying the area with fragments. Of the many missile based CIWS^"^ only RAM- 
HAS^^ BK I missiles are capable of tracking and intercepting waterborne targets. The proba¬ 
bility to hit a target is in excess of P(hit) = 0.95 per missile (Table 2.6). Usually two missiles 
are shot at each target, and deliver a 25 pound blast-fragmentation warhead. The RAM system 
stores 21 missiles per launcher and usually two launchers are equipped on each ship. Therefore, 
it can attack 21 targets until it needs to reload (Diehl Defense, 2009). Even though technically 
possible, most vessels have no extra stock of RAM missiles in their magazines (see Table 2.4). 
To recap, of both gun and missile CIW systems only the newest generation can engage small, 
floating targets with high accuracy.^^ 


^^Including the time to haul the ammo to the gun, reload, and clear the area afterwards. 

^"^To name the most common: RAM (USA/Germany), Sea Wolf (UK), Sadral (France), Sea-Sprint (Canada), 
SA-N-11 (Russia), HQ-7 (China). 

^®HAS: Helicopter, Aircraft, Surface craft interception upgrade. 

^®Or, in case of missile CIWS, only the latest generation. 
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Figure 2.6: Phalanx MK149 Armour Piercing Discarding Sabet reund discarding its sabot and petal (From DRS- 
Technologies, 2004). 


Item 

min 

max 

measure 

ammunition 

9 

30 

tandem missiles 

launcher train 

30 

60 

degrees/second 

target acquisition 

7 

20 

seconds 

inter-firing time 

3 

20 

seconds 

firing arc 

180 

360 

degrees 

engagement distance 

200 - 350 

6000 

yards 


Table 2.4: CIWS (Missile-based) characteristics 


As both systems are nearly identical, with the only exception being that gun based CIWS 
may be reloaded at sea, for this study they are modeled as a single system. The only true differ¬ 
ence is in the cost per engagement, which is not a factor being investigated in this study. 

2.1.3 Helicopters Equipped with AGM-114N Hellfire Missiles 

Most NATO shipboard helicopters are not designed for and are not effective in a surface 
attack role. The U.S. Army is now assigning ground attack missions to fixed-wing aircraft be¬ 
cause ground-attack helicopters have proved to be highly vulnerable to small-arms fire (Grant, 
2006). The only shipboard helicopter designed for ASuW missions is the recently introduced 
MH-60R Seahawk using the AGM-114N and AGM-114M Hellfire II Missiles. In August of 
2008, the first operational Naval Helicopter Squadron equipped with the new MH-60R Sea- 
hawk, accompanied by the AGM-114N Hellfire II missile, became combat operational. With 
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Item 

min 

max 

measure 

ammunition 

2 

8 

Missiles 

Helicopter Speed 

0 

195 

Knots 

target acquisition 

5 

15 

seconds 

inter-firing time 

5 

20 

seconds 

firing arc 

180 

360 

degrees 

engagement distance 

250 

10000 

yards 


Table 2.5: Helicopter with Hellfire Missile characteristics (From Norman Friedman, 2006) 


this missile, precise, effective and lethal firepower may be projected at land and at sea based 
targets, from distances as far as five miles (Lockheed Martin, 2009). 

To find its intended target, the Hellfire missile uses laser guidance. The missile’s laser 
seeker acquires and locks-on to a coded laser’s energy reflected from the target prior to launch. 
The seeker is designed to be resilient against dust, sea spray and light clouds. Once a Hellfire 
loses the laser designator, it will, with a high probability, not reacquire the designator and will 
then miss its target. Training data suggests a P(hit) of 0.3 to 0.7 depending on helicopter crew 
training, weather and target speed (Pike, 2007). If AGM-114M or AGM-114N missiles score a 
hit, there is a very high probability of killing the targeted S AFC. 

Within the simulation there is either no or exactly one helicopter in a scenario. If there 
is a helicopter, it will be armed with two to eight AGM-114N Hellfire II missiles (Table 2.5). 
Zero helicopters in a scenario does not necessarily mean that the frigate has no helicopter com¬ 
plement aboard. This can also indicate that the frigate has a helicopter aboard, but has decided 
not to launch it, or that the frigate has a helicopter, but the helicopter is not armed with any 
reasonable air to ground capabilities. The launch of a helicopter during an attack is thought to 
be a very unlikely event and therefore is not modeled.^^ 

2.1.4 Frigate Weapon Hit Probabilities 

Current hit probabilities for military weapons systems are nearly always considered a 
national secret and therefore impossible to research and implement in a simulation using open 

^^The launch of a helicopter while being attacked is unlikely considering the time to prepare and launch a 
helicopter, as well as the frigate’s steering limitations during the actual launch of the helicopter, both according to 
a subject matter expert. 
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Weapon 

P(hit) min 

P(hit) max 

Main Gun 

0.05 

0.85 

Auxiliary Gun 

0.01 

0.95 

Small Gun 

0.01 

0.98 

CIWS 

0.75 

0.99 

Hellfire 

0.70 

0.99 


Table 2.6: Model used Frigate Single Engagement Hit Probabilities 


sources. Fortunately, some data, originating both in studies on now outdated weapons and gov¬ 
ernment backed research not deemed secret, is accessible to the general public. Therefore, it is 
possible to use this data within this thesis. Some publications such as Jane’s Naval Weapon Sys¬ 
tems (Hooten, 2008) or the Naval Institutes Guide to World Naval Weapons (Norman Friedman, 
2006) publish hit probabilities. For these publications, it is unclear how the authors determined 
them. Companies trying to sell their weapon and targeting systems will often publish reference 
hit probabilities and are another source of information. Also, system builders^* tend to publish 
hit probabilities. Both industrial information sources are usually exaggerated, either positively 
or negatively. Subject matter experts, where available, are a good source on some systems, but 
are often unreliable. Finally there are weapon systems for which no data exists at all. By using 
designed experiments, however, we span a range of settings which is selected as to include val¬ 
ues that, to us, are unknown. 

For this paper, all available open sources are used. Data input is further varied to allow 
for different systems performance and to fully explore the space of possibilities (Table 2.6). 
Factors such as wind, sea-state, crew training, weapon alignment and maneuvering are taken in 
account within the exploration of the model setting, yielding a robust structure for analyzing the 
model. 

2.2 SAFC Setup 

Fishermen in the Bab-El-Mandeb and Gulf of Aden region catch fish using small and 
mid-sized dhows. These dhows are often accompanied by small, agile and fast craft. These 
craft range from skiffs, flatbed boats to small inflatable dinghies. The preferred fishing method 
is either to trawl, or to hunt fish using the small craft. When using the fast craft, speed is im- 

^^System builders usually do not produce weapons themselves, but buy and integrate different systems as a 
package and sell those. 
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portant to steer the swarm into the nets. Therefore, these craft usually get at least 15 knots, but 
modern variants can speed up to 45 knots. 


Especially with the modern Somali buccaneers, a trend to upgrade, as well as to mod¬ 
ernize, can be observed (NATO Operations, 2009). Those full time pirates not only invest their 
ransom money in good living (Arabian Business, 2007), but also in good equipment to get more 
money and, most importantly, in an improved business infrastructure. Early knowledge of tar¬ 
gets is a valuable asset for the pirate business (Newton, 2008). With increased revenues, foreign 
criminal elements are trying to access these business infrastructures, providing additional logis¬ 
tics. These in turn allow the pirates to obtain a very recent operational picture, and are the basis 
of this study’s assumption that the attack on the frigate is planned, not an accident (Tremlett, 
2009). 


Item 

min 

max 

unit 

Max Speed 

15 

45 

Knots 

Size 

5 

15 

meters 

RPG-7 

1 

2 

ea 

Handheld Weapon 

0 

2 

ea 


Table 2.7: Small Agile Fast Craft (SAFC) setup 


2.2.1 Distillation of Different Small Agile Fast Craft Types 

As with the frigates, the SAEC are a distillation of the multitude of vessels considered 
(Table 2.7). Based on observation and media coverage, almost any kind of floating small ves¬ 
sel will be used for fishing purposes, and as such for the occasional pirate activity. There are 
dinghies, inflatable dinghies, flatbed boats, skiffs and various other kinds of glass fabric, metal 
and wood open speedboats. There is not one influential or common type, normally not even 
within one fishing group. Therefore factors such as speed and stability (influencing hit proba¬ 
bilities) will need to be varied extensively. 

2.2.2 Weapons Considered for the SAFC 

Weapon types for the SAEC were chosen based on two assumptions: either because they 
are thought to be readily available on the black market, or because large quantities were sold to 
the area of interest, thus making them accessible to potential attackers. Both assumptions also 
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Item 

min 

max 

measure 

ammunition 

5 

15 

bursts 

target acquisition 

2 

5 

seconds 

inter-firing time 

1 

4 

seconds 

firing arc 

120 

360 

degrees 

engagement distance 

0 

1000 

yards 


Table 2.8: Hand-held Weapons characteristics 


allow the attackers to acquire enough ammunition for the planned attack. Only medium and 
large caliber rifles and shoulder launched weapons are selected for inclusion in this study. Most 
weapons within one category do not differ much in relation to range, lethality and ammunition 
available. Therefore, it is not that important what specific kind of weapon the SAFC employ, 
only what category and how many. Anything smaller than an assault rifle is not considered 
relevant, as the effect on a frigate would be negligible. 

Weapons considered in the “handheld weapon” role include the PK / RPK 7.62 mm, 
the Rheinmetall MG 3, the NSV 12.7 mm, the QJZ 89 (light version), the AK 47, AK 74, the 
H&K G3 and the FN FAL. The main weapon of choice for dealing damage to frigates is the 
well-known “RPG 7” rocket propelled Grenade launcher.The RPG-7 was chosen for both 
market availability and price (Table 2.8 and Table 2.9). 

2.2.3 Handheld Weapon Hit Probabilities 

The hit probabilities for hand operated and aimed weapons, even more so than those of 
weapons mounted on a ship, depend on many factors. There are factors beyond the control of 
the attacker, like target size and speed. Factors under control of the attacker include the stability 
of the platform, speed at which the platform is moving, the weapon and aiming proficiency of 
the user and, only as a secondary measure, the accuracy of the weapon itself.^® 

Single shot and burst fire hit probabilities were collected from subject matter experts 
and, if open sources were accessible and still valid, from army and navy field experiment data. 

^®Each of the listed weapons is produced by many countries, produced for at least 40 years, is officially adopted 
by at least 25 countries and has a very high number built, both licensed and copied (Jones and Cutshaw, 2005). 

^°Both LMG’s and especially HMG’s are not easily trained at targets, but usually have some kind of gun mount. 
In order not to shoot its own crew, and to keep the craft stable, in reality the firing arcs of both weapons are reduced. 
As all handheld weapons of the SAFC are distilled into one this reduced weapon arc is not modeled. 
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Item 

min 

max 

measure 

ammunition 

4 

8 

rockets 

target acquisition 

2 

5 

seconds 

inter-firing time 

10 

25 

seconds 

firing arc 

90 

180 

degrees 

engagement distance 

50 

950 

yards 


Table 2.9: RPG-7 characteristics (From U.S. Army Training & Doctrine Command, 1976)^* 


Weapon 

P(hit) min 

P(hit) max 

RPG-7 

0.20 

0.90 

Handheld Weapon 

0.30 

0.90 


Table 2.10: Model used SAFC Single Engagement Flit Probabilities 


Based on lengthy discussions, handheld weapon hit probabilities for a SAFC are thought to be 
between 1/4 and 1/10 of known hit probabilities, which were experimentally derived by firing 
these weapons at a dedicated range with a stable platform and weapon mounting (Table 2.10). 
Additionally, for series of shots that arrange themselves in chain-like patterns,the simple for¬ 
mula P = 1 — (1 — kpY, where “p” is equal to the probability mass on target for a single 
shot, and “k” is the average fraction of the target area covered by shots, may be used to derive 
the probability of at least one hit out of a salvo of shots (Zahle, 1971). These probabilities are 
exponentially adversely influenced by sea state, from no influence at sea state 0, to a 35 percent 
decrease at sea state 1, 75 percent at sea state 2 and 95 percent at sea state 3. Small craft are not 
likely to operate beyond sea state 3 (Associated Press, 2009). These factors are included in the 
design of experiments. However, since they are outside the control of NATO Forces, they are 
only interesting insofar as they contribute to the uncertainty of the outcome in an encounter. As 
such, they are only modeled at two levels. 

2.2.4 Weapon Hit Probabilities versus Helicopter 

No dedicated hit probabilities are designated for attacking the helicopter. As the frigate 
is the primary target, the helicopter is modeled so as to usually operate from out of reach of the 
SAFC, which is quite a realistic assumption. This way, the SAFC will focus its attack on the 
frigate, and only attack the helicopter if they happen to be near it by chance. 

for example with an automatic rifle/gun employing belted ammunition and a low inter-firing time. 
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Figure 2.7: A variety of SAFC weaponry (not to scaie, ciockwise from upper ieft): NSV 12.7, AK 47 II, H&K G3A5, 
Rheinmetall MG3A3, RPG-7. 


2.3 Scenario and Modeling Environment 

This section describes how the MANA modeling environment, Version 4.04, was uti¬ 
lized to develop a green water scenario of a frigate being attacked by multiple small, agile and 
fast craft. After a brief overview of the model’s capability, the advantages and disadvantages 
of using MANA versus other modeling environments are discussed. The impact of using time 
stepped versus discrete event simulations is discussed. Finally, the model implementation and 
the chosen measures of effectiveness are explained. 

It is important to remember that this study is interested only in whether the frigate sur¬ 
vives an engagement or not. The survival rate of the SAFC and the survival of the helicopter if 
present in a scenario are not of interest. 

2.3.1 Attacker Motivation 

The motivation of SAFC is to achieve maximum media coverage of the incident and 
a disgrace of the stronger and technologically advanced enemy. The attackers hope for some 
form of retaliatory measure involving declared innocents, therefore gaining more followers or 
money to continue their aggressive behavior and expand their influence. 
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As the attack occurs at sea, for best effect the attack must either be recorded or the 
attacked ship needs to be able to return to port, exploiting full media coverage. This will be 
considered when determining the number of rounds each SAFC fires before it flees. 

2.3.2 Area of Operations 

The area of operations (AO) is set within a free stretch of water with no barriers or 
boundaries. The frigate is on an assumed typical intelligence gathering and observation mis¬ 
sion. There are no other friendly forces but some assumed neutral traffic in the AO. There are 
no known hostile contacts in the AO, and no intelligence of planned hostile activities is known. 
The frigate operates under standard NATO ROE’s. One noteworthy ROE is that the frigate may 
actively defend itself when being shot at or when the attackers cross into the frigates safety zone. 

2.3.3 The Chosen Agent-based Modeling Environment: MANA 

Many modeling environments were considered when planning this study. The MANA, 
Pythagoras and SimKit modeling environments were each considered to determine how well 
they were suited for the study. 

A “model” as used in this study is defined “as a system used as a surrogate for another 
system (Schruben, 1992).” In a typical agent based modeling environment, a system of agents 
“substitutes for human subjects and military hardware in the experimentation process and are 
the sources of the data (Alberts, 2005)”. An “agent” in context as used in this study is a net¬ 
work of interactive processes and elements (Holland and Miller, 1991). The environment is the 
software, spreadsheet or any other means within which the agent based simulation is executed. 
An Agent Based Simulation (ABM) is thus “a simulation made up of agents, objects or entities 
that behave autonomously. These agents are aware of (and interact with) their local environment 
through simple internal rules for decision-making, movement, and action (Cioppa et ah, 2002).” 

The first environment studied was SimKit. SimKit is a discrete event simulation applica¬ 
tion program interface (API) for creating models using Java. Building a model in SimKit starts 
by developing an event graph (Schruben, 1992) of the agent behavior to be modeled, and then 
implementing this event graph in SimKit (Buss, 2009). After development and initial imple¬ 
mentation of the event graph (Appendix D), SimKit was not pursued any further. Even though 
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SimKit allows implementation of the model in great detail and exactly to specifications, the 
time to build a running model and to implement correct agent behavior was deemed too costly 
when compared to the time needed for actual analysis of data gathered by the model. The event 
graph was used for the implementation in the other modeling environments, as it is a great tool 
to visualize what is to be studied. 

The next environment investigated was Pythagoras. Pythagoras is a time-stepped agent 
based simulation that was developed for the United States Marine Corps’s Project Albert (Lawlor, 
2005). Using Pythagoras it is possible to model soft decision rules, driving the agents by desires 
and distinguishing between different parties of the overall same side using a concept called “sid¬ 
edness” (or affiliation). Pythagoras software has been released as open source (Stallman, 2002) 
and is designed to produce modeling results quickly and easily. Pythagoras was not chosen as 
the modeling environment for this study as it has no concept of agent orientation. Modeling 
port and starboard is an important concept for frigates. Frigate weapon systems are typically 
built symmetrically about the sagittal plane, resulting in complementary weapon arcs (Bitinas 
et ah, 2005). 

The model chosen was MANA, the Map-Aware Non-uniform Automata. MANA, cur¬ 
rently at version 4.04, is actively developed by the New Zealand Defense Force. The developing 
agency is the Defense Technology Agency (DTA) (New Zealand Defense Force, 2009). At time 
of writing, version 5 of MANA is in beta testing, and is expected to be released around the 
beginning of 2010. MANA is a time stepped, agent based modeling environment designed with 
a graphic user interface. MANA was designed to be user friendly and to achieve fast run times. 
MANA is delivered as a pre-compiled executable for any Windows based environment. MANA 
has built-in data farming capability based on an XML input file, allowing one to implement a 
design of experiments quickly and efficiently once the XML syntax is well understood. Agent’s 
desires are easily adjusted and state changes to their behavior are well implemented. By hosting 
and participating in International Data Farming Workshops, the NFS Simulation, Experimen¬ 
tation and Efficient Design (SEED) Center for Data Earming has direct ties with DTA (Horne, 
2009). This allows for direct consultation and rapid problem resolution with DTA. MANA was 
chosen as it incorporates all aspects that were deemed necessary for a successful implementa¬ 
tion of the model for this study. 
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2.4 Model Implementation in MANA 

Whatever is necessary to answer the research question, a good vision of what needs to 
be implemented and measured is crucial in the process of building the simulation model. In 
case study, the behaviors and interactions were visualized using an event graph. Alas, as with 
most prefabricated modeling environments, it is not always possible to convert vision to model 
in a one-to-one fashion. Sometimes it is necessary to think around predefined edges, bending 
the model, and some things can not be implementable at all. 

As is true with battle plans, a vision will not survive its first model runs unchanged. 

2.4.1 Battlefield Map 

The battlefield map represents open waters, with no natural movement constraints for 
any of the agents. The frigate agent’s starting area is in the lower left comer. The frigate agent’s 
goal is in the upper right corner. 

Underlying the map is a grid of 600x600 pixels. Each grid is designed to represent a 
square with a side length of 50 yards. MANA is time stepped, and the fastest agent, the heli¬ 
copter, has a maximum speed of 190 knots. A timestep is translated to represent 0.5 seconds, 
which prevents the helicopter from skipping over grid positions.All agent movement and any 
actions taken therefore need to be scaled for compatibility with the selected timesteps. 

2.4.2 Initial Settings of the Model 

The scenario places the frigate on a straight track from the lower left to the upper right 
comer of the map. As described in Chapter 3, the track will be traversed at different speeds, 
allowing the frigate’s contact with the SAFC to occur at different times, speeds and bearings. 

The SAFC are randomly instantiated in the two yellow areas (Figure 2.8), whereas the 
frigate’s instance (and the helicopter) will be randomly placed within the green area of the map. 
The frigate will initially follow the green track through the map (Figure 2.8). 


^^Skipping grid positions is generally considered a bad thing, as it makes it possible to pass through barriers and 
teleport through agents occupying a given area without detecting and / or interacting with them. 
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Figure 2.8: The basic setup of the battlefield map. 

There will always be only one frigate, either with or without an aoeompanying heli¬ 
copter, and between four and eight SAFC. The SAFC will track and try to intercept the frigate, 
but will also attack the helicopter if offered the chance. The helicopter will either be available 
from the start of a scenario, or not at all. 

2.4.3 Conditions of the Model 

The frigate will be considered to have “lost” the scenario when it is hit as many times as 
it has allocated hit points. Using this method allows us to conceptualize the frigate as a single 
entity without the need to model every and all compartments, personnel and machinery. 

The attackers are not evaluated regarding “lost” or “won” conditions. Attackers are 
designed to expend all ammo of their main weapon, trying to escape afterwards. An escaping 
SAFC is designed such as not to be a target of interest to the frigate. 
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2.4.4 Constraints of the Model 


MANA is, essentially, designed and optimized for land combat modelling (McIntosh 
et ah, 2007b). Therefore, many agent’s behaviors are not modeled as would be expected from 
a naval perspective. The behavior lacking the most is weapons control and weapon assignment. 
Each agent can use up to six weapons, numbered one through six. Weapons are not fired truly 
simultaneously, even though they may be fired in the same timestep. MANA uses a simple 
algorithm for determining what weapon to use to attack a certain enemy agent. According to 
the MANA manual. 

Weapons in MANA are modeled using simple probabilities to decide if agents 
can been shot within a certain range.... (E)ach turn, each weapon with ammu¬ 
nition is fired. Eiring is in order of weapon number. The user is advised to put 
their most effective weapon first (number 1) to avoid wasting ammunition from 
less effective weapons. (McIntosh et ah, 2007a) 

Thus the algorithm does not distinguish between hit probabilities (Eigure 2.9). Extensive testing 
has also shown that an agent will within one timestep often not discharge more than one weapon 
at a distinct target. More interestingly, this behavior is observable over multiple timesteps, even 
if more than one weapon is within range and within the firing arc during all timesteps. This 
is caused by the probabilistic method to determine if a certain weapon is discharged during a 
certain time step. Most of the weapons shoot once every twenty turns, on average. Given this 
input, the probability that they will not discharge in a given round is 0.95. The chance that a 
certain weapon will not have discharged in 30 rounds is (1 — P)^°. 

As it is not possible to influence the discharging algorithm, the hit probabilities per se 
have implications on modeling. The agents will shoot their weapons even when probabilities to 
hit are very low, for example at a long distance. An experienced or well trained human operator 
would probably not shoot and waste ammunition at those distances. Therefore, weapon ranges 
are reduced to avoid this undesired behavior. Weapon ranges are cut off at a predetermined max¬ 
imum effective range. Similarily, the decision was made not to vary weapon position within the 
design panel. Eor the frigate, weapon numbers are as follows: 1 — Main Gun; 2 — Aux Gun 
Port; 3 — Aux Gun Starboard; 4 — Small Guns; 5 — CIWS. Eor the SAEC, weapon numbers 
are as follows: 1 — RPG-7; 2 — Handheld automatic weapon; 3 — RPG-7. The Helicopter 
only has one weapon system, the Hellfire missile, always at position one. The calculation of hit 
probabilities corresponding to the timestep is detailed in Chapter 3. 
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2.4.5 Constraints of Modeling 


In order to intercept the frigate, the SAFC will always know the position of the frigate 
and never lose track, as may happen in reality. Also, the frigate will only employ up to six 
different weapons at one time, as the model only allows this many weapons to be equipped 
at once. Multiple identical weapons other than the auxiliary gun systems are modeled as one 
weapon with a higher P{hit), as more shots are fired in the direction of the enemy than with a 
single weapon. 

Two distinct behaviors of the frigate are modeled: with and without regard to ROE. 
Without ROE, once one SAFC is deemed hostile, all other approaching SAFC are also classi¬ 
fied as being hostile. This may actually reflect the reaction of the frigate in a real engagement. 
In the heat of battle, most contacts closing in and somewhat similar to the hostile vessel would 
be assumed to be involved with the attack in progress. 

When on the other hand the frigate is acting in accordance to ROE, it will only shoot 
at targets that either shoot at the frigate or get too close to the frigate. “Close” is a factor that 
is varied extensively, modeling different types of commanding officers. To prevent the frigate 
from shooting at fleeing targets, they will change to a different state, and be effectively neutral. 
The frigate will then at once cease firing upon the SAFC, even though in reality, more shots 
might be fired until the target is no longer deemed to be a threat. 
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Figure 2.9: MANA setup panel for a high explosive weapon. 
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2.4.6 Damage Model for the Frigate, Helicopter and SAFC 

The frigate has a fixed number of initial hit points. After the frigate has sustained a 
number of hits corresponding to the number of hit points, it is considered to be killed, and the 
scenario is over. This kill represents a mission kill. The number of hit points allocated to the 
frigate is an abstraction and represents three different modeling aspects. First, it represents the 
structural integrity of the frigate, with all its components. Secondly, it represents the crew’s 
training and aptitude to repair damage sustained. Third, it represents the size of the frigate. A 
frigate is comparatively large; it is very hard to actually hit it where real damage may be dealt. 

The model developed does not allow for catastrophic failure on the part of the frigate. 
This decision is based on two assumptions: 1) the probability of a catastrophic hit on a frigate 
is not predictable (as there are few data available on hit probabilities, and there are no data on 
how fuel or explosives on board the frigate react to small arms fire) and 2) including catas¬ 
trophic failures does not add to the analysis of this experiment. Even though the set of unit 
kills contains the mission kill, it might add undesirable interactions. For these two reasons, a 
catastrophic failure is not permitted. 

Allowing SAFC to have hit points may seem awkward. When determining a hit on a 
SAFC by, for example, a frigate’s 127mm main gun, it may seem wrong, or strange, to allow 
the SAFC to continue to operate. On the other hand, a hit on a SAFC is not necessarily a direct 
strike. When looking at it in this way, allowing the SAFC to have hit points makes sense. Con¬ 
trary to the frigate, the SAFC hit points represent the inability of the frigate’s weapon systems 
to accurately track and engage these small and fast vessels, especially in higher sea-states. 

2.5 Measure of Effectiveness in MANA 

A “measure of effectiveness” is a common measure in which vast amounts of data are 
distilled into relevant information related to a specific objective. As such in this study it is used 
as a performance indicator. The frigate is attacked by different numbers of varying types of 
small agile and fast craft. It was planned to measure the defender’s effectiveness by the formula 
“damage obtained divided by attackers neutralized.” This measure is not supported by MANA, 
as the software does not provide feedback on damage dealt or received. 
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Therefore, it was decided to use the number of frigates killed^"^ as the measure of effec¬ 
tiveness. This binary response variable is then averaged over the iterations and over the noise 
factor settings, effectively converting the response from binary to continuous. 


only one frigate is modeled, this value obviously is either zero or one. 


36 



CHAPTER 3: 
Design Of Experiments 


3.1 Introduction 

Design of Experiments (DoE) describes the setup of a simulation’s variable parameter 
space. In reality this variability may or may not be under control of the designer (Eisher and 
Prance, 1974). Design of Experiments (DoE) follows a stringent methodology. DoE for sim¬ 
ulations employs elements such as randomization, replication, orthogonality and experiments 
whose design consists of two or more parameters with discrete possible values each, known as 
“factorial designs” (NIST, 2009). 

Within this thesis a fractional factorial design will be developed to answer the questions 
as discussed in Chapter 1. Eollowing a brief discussion of definitions and formulas as used in 
the following sections, the variables varied in the frigate and SAEC design are detailed. Then 
it is explained how the experiment is set up. Einally insights that were discovered during the 
execution of preliminary model runs detailed. 

3.2 Definitions and Formulas 

3.2.1 Data Farming and Data Mining 

In this thesis, a technique called “data farming” is employed. With this technique a sim¬ 
ulation model is run over thousands or even millions of replications. At each replication, input 
variables are changed using a predefined, designed approach (Horne and Meyer, 2004). This 
data farming approach “allows a scenario’s parameter space to be explored rapidly (McIntosh 
et ah, 2007b),” and helps to identify regions containing interesting constellations (Cioppa et ah, 
2004). Using a specific experimental design ensures that the parameter space is searched in an 
efficient manner. 

Data farming is not to be confused with data mining. Data mining is the science of ana¬ 
lyzing large data sets or databases with many different algorithms and procedures to find some 
information previously unrecognized within all the available information (Hand et ah, 2001). 
Thus, data mining will take any amount of data that is collected for varying purposes and ex¬ 
tracts information, whereas in data farming through design of experiments resulting data seams 
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are modeled a way that information might be easily harvested by employing a predetermined 
procedure (O’Reilly, 2005). Using a data farming approach data mining may nonetheless be 
used to discover relations in the produced data stream that may otherwise be overlooked (Ku- 
siak, 2006). 

3.2.2 Robust Design and the Taguchi Method 

Often, only parameters are admitted to a design of which the designer — sometimes 
with the help of some subject matter experts — believes are valid. They actively try to suppress 
any external variability when testing their design. Sometimes, this can result in assumptions 
that greatly underestimate the variance of certain events. To avoid this caveat a methodology 
called “robust design” will be used in this thesis (Taguchi, 2002). This method allows the exper¬ 
iments designer to determine which parameter values most affect the measure of effectiveness 
in a chain of experiments. This is achieved by both varying parameters that are influenceable by 
the design specifications, as well as those that are encountered under real conditions, and typ¬ 
ically are not controllable. All these variations are then tested in a multitude of combinations, 
whether they might or might not occur in practice. This extensively regulated process is called 
the “Taguchi Method” (Taguchi, 1995). While in this thesis we will vary both controllable and 
uncontrollable parameters extensively, strict adherence to Taguchi’s tactics are not required for 
a robust design. Therefore, Taguchi’s strategy is followed, but not his tactics. 

3.2.3 Controllable Factors and Noise Factors 

Primary treatment variables or any input parameters and structural assumptions com¬ 
posing a model that could potentially be changed are called “factors” (Law, 2007). Factors are 
classified as controllable and uncontrollable factors. A controllable factor is one that may be 
influenced by the decision maker, e.g., the number of guns on a ship. Uncontrollable factors 
are those not controllable by the decision maker, e.g., the weather or an adversary’s weapon 
systems (Kass, 2006). Uncontrollable factors may be treated as “noise factors”. Even though it 
is accepted that it is not possible to influence their value, they might be relevant to the behavior 
of the system. Most often it is possible to predict a valid range in which they might be en¬ 
countered, e.g., by assessing historic data. As all factors in the experiment are accessible to the 
experimenter, they may be varied and thus introduce natural variability to the model. In contrast 
to controllable factors, they are not treated as input parameters to the later analysis (Law, 2007). 
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3.2.4 Design Points 

A factor may be static or have different levels during the run of an experiment. When 
constructing an experiment with multiple varying factor levels each factor will be combined 
with other factors. This is usually done in a manner such as to reduce correlation. These factor- 
level combinations are called design points (Banks et ah, 2006). 

3.2.5 Step Function, Concerning Hit Probabilities 

In models not using physical representation,^^ a true (or more drastically, real) hit prob¬ 
ability is nearly impossible to calculate due to the high level of abstraction. In contrast to 
approaches using highly complicated formulas to determine hit probabilities, it was decided to 
use a step-function to model this problem. 

A step function results in hit probabilities that remain constant between a minimum 
distance^^ and a predefined maximum effective distance. This maximum distance is called 
maximum effective range (MER). MER is the “maximum distance at which a weapon may be 
expected to be accurate and achieve the desired result (Navias and Hooton, 2001)”. It is as¬ 
sumed that a human controller proficient with a certain weapon would know about the rapidly 
declining accuracy and thus not shoot further than this distance. 

3.2.6 Data Conversion for the Model Implementation 

Conversion of speed, rounds per minute and inter firing times are calculated by formu¬ 
las. Speed is the most complex formula, as distances are measured in basic grids of 50yds per 
timestep, and the corresponding timestep is half a second. The formula for converting MANA 
speed expressions to expression of speed in knots:^^ 

modeling atoms, molecules and their interactions. 

^^This is a physical constraint due to a gun’s barrel position combined with its maximum elevations, and an 
explosive projectile’s fuses arming only after some flight. Compare Joint Publication 1-02 (Navias and Hooton, 
2001). 

^^This conversion is not quite correct, as one knot actually converts to approximately 2025.371 yards per hour. 
As the model of this study is based on quite a high level of abstraction, dropping about twenty six yards per knot 
per hour is deemed acceptable (Institute for Electrical and Electronics Engineers, 2007). 
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,, (60 * 2 ) , , , 

* 60 » 50 )/ 2000 . 

MANA represents speed by the expression Pmove = {Grids/lOO) per turn, where Grids 

e N < 1000. 


Rounds per minute, a measure often expressed in weapon system guides, are eonverted 
from MANA (and viee versa) as follows: 
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Another important measure to have a real world expression for is inter firing times. 
Inter-firing times are a measure for how long it takes between shots at a eertain or two different 
targets. Inter-firing times are eonverted to and from MANA using the following formula : 


'(i/( 


InterFiringTime \ 
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Inter firing times are represented in MANA by the Expression Pshoot = {Shots per turn. 


3.2.7 Nearly Orthogonal Latin Hypercubes (NOLH) 

NOLH History 

NOLH experimental design teehniques were researehed and implemented by Lt.Col. T. 
Cioppa in 2002 and further refined by Joshua Ang Keng-Ern in 2006. Lt.Col. Cioppa began 
his work by relaxing Orthogonal Latin Hypereubes (OLH) design eriteria. He eoined the term 
NOLH and researehed these Latin Hypereube subelasses to effieiently work with simulations 
that require a huge number of input faetors. At the same time, NOLH designs require only a 
limited amount of prior assumptions (Cioppa, 2002). 

The nearly orthogonal arrangement is designed in a way that a maximum pairwise eor- 
relation of 0.03 is observed. The nearly orthogonal quality of the design allow for addition or 
removal of terms without affeeting the estimates, whieh in turn make the use of effieient analy¬ 
sis methods possible (Ang Keng-Ern, 2006). This property in eontrast to traditional grid-based 
faetorial design teehniques, whieh either eonsidered only high and low values for the faetor lev- 
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els or had to constrain the number of input variables in order to complete the simulation study 
in a reasonable amount of time. 

Desirable NOLH Characteristics 

From a design of experiments point of view, the orthogonality as measured by the ab¬ 
solute maximum pairwise correlation pamp and the condition number condi^M) is very desir¬ 
able. Another desirable characteristic of interest is the design’s space filling characteristic ML 2 
as measured by the formula (Hickemell, 1998):^^ 

/ ^ n n k 

ML2(X) = ( - j - —- max(a;rfi,a;,i)]. 
k / d=li=l ^ d=lj=li=l 

This measure is important, as it allows the fast exploration of huge factor ranges that otherwise 
would have to be explored by computing every possible factor-level combination. That kind of 
design is called a full factorial design. A full factorial design with up to five or six factors is 
computable in reasonable time. 

NOLH used in this Study 

NOLH designs will, due to their design characteristic, not explore the complete possi¬ 
bility space. This is best shown when comparing the scatterplot matrix of the frigate’s factors 
(Figure 3.1). They will allow the analyst to explore — in finite and reasonable time — a larger 
and still very broad section of this input space as in comparison to traditional methods. To gen¬ 
erate the NOLH design for this study, a spreadsheet coded by Professor Susan Sanchez, NPS 
Monterey, is used (Sanchez, 2005). 

3.2.8 Factor Lockstepping 

Due to the complexity of finding NOLH designs of high order, only models containing 
up to 29 factors may be modeled using the spreadsheet at this time. The final experimental 
model of this study contains 41 individual factors. To study these factors with the available de¬ 
signs, some factors were lockstepped. Lockstepping is the process of varying multiple measures 

^^ML 2 {'K) is a variant of the Lp-discrepancy where X denotes the matrix being measured. Xdi, Xji denote any 
two vectors of X. n and k are width and depth of X, and d is a counter. 


41 






^ ■ 



'si 

ill 


Itt 

M 

■- 





^11! 




B 

B 



rl 

1 




VI 




H 





!»■ 




m 



If®; 

m 

..'J 



»? 






“ 

■V 

8S 


" 

• s 

- 


m 

m 



‘ 

» 



■ 

m 

— 

8 1 


liia 

m 



i 

m 










m 


&M 



-■ 

il 



m 




m 

m 





— 























- 







_ 



. 1,:= 

- 

-- 

- ■ 












== 





— 

—r 

— 



... 



= 

=- 



“Z 

■= 

— 


' .-. 

_ 


“ 

• ■ 

— 

— 




— 


















— 

— 



— 

— 

— 








i 

i 

li 

-1 

id' 

ii 


'! 1' 

IN 

113 

li 

i;?, 

1 



III 

1 n 

'M 

iff 

I III 

ill 

,v* 

if 


1 

n 

1 

15 

'iii' 


iii 

■ ! 

1 


§9 

'A 

1 

t 

It 

•i 1 

i'll 

|i« 

-■ 

liiii 

mi 

ll» 


■■■■ 

1 


m 

ill 

'h 

* 'll 

ilpiji: 

',i!| 

II 


tit 


1 

ii 

f III f 

III 


.J 

Ill 

1 


hi 

h. 

1 


-1 




i* 

----j 




- -i 











--- 

- 


:: r 

r 

z : 


iJL 





-- 

:--- 




... 



— 

— 


■ ■ 












— 

— 

— 






— 

— 



— 

— 

— 



— 

_ 












lZ 

. 













_ 

_ 



_ 

_ 

_ 


:: 




TT 




1 

iHji'-. 

ii 






p" 

Z 

Z 



■kj '5' 


U 


«( 

,''ii 



T'' 


'IV 




... 






s 

s 



- 

kfi 





Ill 

ml 

i:i 

III 

r^’kV' 




n 

.1 

I.* 

m; 


IS 


1 











if-. 




o 




m 

'ill'lis 


. ; 



'X: 


-jy. 


„ s 


« « 


/« 

l;s 










— ■ 

























— 

— 



— 


— 








































1# 


ill I 

iS 

fi 

i|ii 

i 

IIS 

m 





mi 

Hi 

' 

ii'! 

1 

it 



i 1 

1 

C'i 

il 


ii'ii' 


m * 

I'l 

j 





, ill 




IIK 

m 

liiiili 



m 

r,;, 









I'X- 

- 











a 

a 

M 







\\m 




1 jffi 
















-i 



WM 




Z". 

MM 

m 





'Y. 


II® 





& 

-■ 



a 

1 !: 





m 

m 



Jf 

vi 

|S 

il ii 






“A'r* 






Ill 












Ill 



..... 



S- 



z z 

z 

z _ 


=^’-. 

--- 





--- 



— 

: = 

” ■ 



= 



— 

— 




F= 






— 

= 

— 



-- 

.. 


= 

=■ 



— 

= 

IZ= 


i 

1, ■ 


1% 


ii 11 

im 


llll 

II 



il 

1 

1 

S 

11 

IS; 

i,:: 

I; 

I 

'III 

'III 

— 

il? 

'd? 

1 

II 

’'I 

if 

!'P 


Si? 

III 

1 

Id 


■« 

1 

e. » |( 

% 

h? 

:;l 

i'l II 

iia 

s? 

. 


w 


rm 

1 

1 

m 

ii 

llll 

:);ii|| 

i: 

1 

1 




1 

li 

fi 

p 

'I'l 

> Ml 

S- 


III 

! 

u 



1 

1:'; 

§ 

& 

d 

ill' 

ii^/ 

1 


ill 

II 

j 


li 

1 

1 


il 

Vv 

'PI. 

III 


III 


9 

— 

1 

(1 

:| 

if 

|ll!l 



III 

■III 

I 




I 1 


— 

— 



— 

— 


















-- 

-- 

-- 


— 

— 



— 

— 

— 


i« 

1 

ms'i 

1 ■ 

1 ! ?«?•« 


l!!llll' 
i|' 1 

Mil 

III 

1 ‘■r « 

•y-y 

■ igi 

; 


XV* 

S«'c 

I'll ill 

* 


nil! 

mu 

: 1 1 1 
: 1 1 1 




1 

s Si 
» 

s; ^ 

- « 

■# > 
B 


IIIIII 


sT-Sy 

Mil 

I 1 1 1 

1 




1 1 

. iiiii; 

Sfi 

i"TTr 




lllillll 

1 1 ' 1 

1 1 1 1 

; 1^ 



1 


. i- X 


.|.!iM.. 

tL.. 

ii" 

mil 

'III 

1 


ssm 

J 

1 

~ te 
» St 

e £! 

mill 

mu 


1 1 

! 


msf 
r ; 


1 1 

t 


rsf 

1 SS4 

: 1 

III! 11' 

1 " 

11%^ 

w’- 

h III' 

11:1 

llll 

“ 


'K'ifSit 

S ill 




; 

if !$ 

III! Ill 

1 1 

;:if 

mil 

mil 

'III 

llll 

'ism 



I 


mill 



.III 

1 


/£ 

SPvts 


1 1 



; = 

.: 







--- 












=. 


E_ 

5 


— 



ITI 


.8 






jii 

«l 

mil 

\' 

M <V 


pliim 

II 

Iw 


- 



s: 

II 

, Il 

III 

ii 

III 



is 

1 

f 1 

;iiij 

. 



if 



1 


« 

irs 

•>'1 

III 1 i 

lisy 


iiiiii 

III 

\\m 

K 

il 

1 

1 


III 

iliiililil 

ii 

I'ii 

il 

-iT 

y-f 


1 

Jl 

•rlli 

,'|i 

% 

- 

m 

1 

’A 


M 

I 1 




:= : 



















== 



” = 

= r 



= 

— 



_ 




— 


— ■ 





















-- 

-- 

-- 


— 

— 


— 

— 


— 




IS 

i'l 

il : 

i;3i 


li'" 

II 

IS 

i’t 

C yi 
yV 



ff 

ill 

ill 


! ill 

'II 


s 

- 

1 

II 

ri ^ 

if 

'1 i!' 

J ,1 


© 

' 

1 

— 


.ft. 

1 

E III 

i 

ifef 

1 

i| i 

13 

S 

il. 

1 I 

s 


ill 

1 


m 

il 


lUllii 

"I'l 

'il 

il 





il 


il 

il i| 

y, .J 


III 

j 

M 

—■ 

Si 

1 

“ -iiii:;!' 

Si 

li 

SI 

mil 

i4 

J jr 


llll 

II 

l';c 

m 

'7i' 



¥% 


iiliiilii: 

ii' III 

id' 

iiii 

III 



G? 

1 

li 

■4 

I? 

C" 

v'q 

ii- 

*4'“ 

I'i 


M 



1 






































Figure 3.1; Scatter Plot Matrix of the frigate’s variables to illustrate both the orthogonality and space filling properties 
of the NOLH used in this study. Names of the variables are in the diagonal. 


using a single multiplier variable. Looking at Figure 3.1, rows 5,6 and 7 are easily reeognized 
as being lockstepped. 


The eontrollable faetor matrix C is ereated by eombining the non-loekstepped faetors 
with the lockstepped factors calculated using the multiplication factors. The tables in the “Con¬ 
trollable Factors” (Chapter 3.3) section indicate the use of a NOLH or multiplication factor 
controlled variable. 
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3.2.9 Hadamard Matrices 


Hadamard matrices are named after the French mathematician Jacques Hadamard, in¬ 
vented in 1867. A “Hadamard matrix is a square matrix whose entries are either -i-l or -1 and 
whose rows are mutually orthogonal (Mood, 1946)”. A Hadamard matrix El of order n satis¬ 
fies:^^ 

El ■ El"*" = nin 

thus, consequently the determinant of El equals Suppose that M is a complex matrix 

of order n, whose entries are bounded by |Mjj | < 1, for each i, j between 1 and n. Then 
Hadamard’s determinant bound states that 

|det(M)| 

Equality in this bound is attained for a real matrix M if and only if M is a Hadamard matrix. 

The order of a Hadamard matrix is either 1, 2, or a multiple of 4. The Hadamard conjec¬ 
ture proposes that a Hadamard matrix of order 4/c exists for every positive integer k (Cameron, 
2006). 


In this paper Hadamard matrices are used to induce factor variance with a two level 
noise factor design. The -i-l entry is used for the high, and the -1 entry for the low level value. 
As Hadamard matrices are orthogonal, with a pure Hadamard design there is no correlation in 
between distinct factors. 

3.2.10 Crossed Design 

In a crossed design, each design point of one design matrix is concatenated to all rows 
of a second design matrix. For a robust design, a high-resolution design C such as the NOLH 
to study the controllable factors, and a low-resolution design N such as the Hadamard matrices 
to study the noise factors is used. This results in a run-file matrix M. M is saved in the form of a 
comma separated values (CSV) file. CSV files are human-readable, standardized text-files. 


^^Where /„ is the nn identity matrix and is the transpose of H. 
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3.3 Controllable Factors 

In this section all controllable parameters are explained, whether they are factors di¬ 
rectly controlled by a NOLH design, factors created by computation through lockstepping or 
fixed parameters. 

3.3.1 Frigate Factors 

Factors modeled for the frigate are its hit points, speed and the actual weapon mix em¬ 
ployed by the frigate in each model run. Part of this weapon mix is the frigate’s helicopter. 

Frigate Hit Points 

As shown in Chapter 2, a frigate is a complex system. For this study, most of the com¬ 
plexity is modeled by hit points. The hit points represent three major concepts. First, they 
represent the actual sustainable damage of the frigate’s hull. Second, they represent how well 
the frigate’s crew is able to control for any damage sustained. Finally, they are a measure of 
how probable a catastrophic hit is believed to be. As the frigate itself is a distillation of many 
different frigates, this value is varied extensively, and is implemented by an NOLH design. The 
minimum value of sustainable damage is 15 hits, and the maximum is 30 hits. Any damage 
sustained is not healed (repaired) during a run. 

Frigate Speed 

Once set, the frigate’s speed is held constant during each run. This factor is varied by a 
NOLH design. The minimum speed modeled is 14.4 knots and the maximum speed modeled is 
34.2 knots (Table 3.1). Speed is not represented in whole numbers as MANA input parameters 
are not entered as knots. One turn in MANA is defined as .5 seconds; therefore, the resulting 
real-world numbers are not whole."^° 

Frigate Weapon Mix 

The weapons used by the frigate model are determined by a “weapon pack,” varied via 
NOLH from 1 to 5. Each number is associated with a certain weapons pack; all packs equip a 

“^'’Which, for the real world, is very real. It is actually very hard to exactly move at a certain speed — therefore, 
it is not deemed important that these numbers do not represent whole numbers. 
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Item 

min 

max 

measure 

varied by 

Sustainable Damage 

15 

30 

hits 

NOLH 

Maximum Speed 

14.4 

34.2 

knots 

NOLH 


Table 3.1: Frigate factor design 


main gun. Pack 1 adds a small gun; pack 2 consists of a small gun and a CIWS; pack 3 contains 
an auxiliary gun; pack 4 an auxiliary gun and a small gun; pack five adds an auxiliary gun, small 
gun and a CIWS. If a weapon is not enabled in a certain design point, its ammunition will also 
be reduced to 0 (Table 3.2). 


Weapon Pack 

Main Gun 

Aux Gun 

Small Gun 

CIWS 

Pack 1 

1 

0 

1 

0 

Pack 2 

1 

0 

1 

I 

Pack 3 

1 

1 

0 

0 

Pack 4 

1 

1 

1 

0 

Pack 5 

1 

1 

1 

I 


Table 3.2: Frigate weapons packs — 0 denotes not present 


3.3.2 Main Gun Factors 

All frigates are modeled with having exactly one main gun. The main gun has six vari¬ 
able factors (Table 3.3). Factors modeled by a NOLH design are the main guns hit probability, 
ammunition, firing arc and its maximum effective range. Minimum range, inter-firing times and 
the round’s impact probable lethal range (hit radius) are represented by dedicated scale factors 
modeled by NOLH. In multiple preliminary runs, these factors were shown not to be significant 
predicting damage to the frigate. 

Main Gun Hit Probability 

This factor describes the hit probability for the main gun’s projectiles. It is held constant 
in between the main gun’s minimum and maximum effective range. The hit probability is varied 
between 0.05 and 0.85. 
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Main Gun Ammunition 


This factor describes the amount of ammunition for the main gun as carried by the 
frigate. The amount of ammunition is varied from 15 to 45 rounds. 

Main Gun Inter Firing Time 

The gun associated inter firing time. This factor subsumes such factors as reloading, 
target acquisition and aiming. The frigate’s main gun is modeled shooting between 2.4 and 8 
rounds per minute. 

Main Gun Firing Arc 

The firing arc of the main gun. This factor describes the physical and superstructure 
constraints of the weapons placement aboard the ship. The firing arc has its center at the bow 
of the frigate, and is divided in equal sections to both sides. The firing arc has a minimum value 
of 150° and a maximum value of 270°. 


Item 

min 

max 

units 

varied by 

Hit probability 



P(hit) 

NOLH 

Ammunition 

15 

45 

bursts 

NOLH 

Inter firing time 

2.4 

8 

rounds/minute 

scale factor 

Firing arc 

180 

360 

degrees 

NOLH 

Lethal Radius 

50 

150 

yards 

scale factor 

Maximum effective range 

2500 

5000 

yards 

NOLH 

Minimum effective range 

150 

350 

yards 

scale factor 


Table 3.3: Main Gun factor design 


Main Gun Lethal Radius 

The distance from point of impact around which lethal fragments are distributed. Many 
different sizes and types of rounds are used in naval warfare. Against SAFC, lethal radius is 
varied between 50 and 150 yards. 

Main Gun Maximum Effective Range 

The maximum range at which an efficient use of the expended round(s) is to be ex¬ 
pected. Against SAFC, the MER is varied between 2500 and 5000 yards. 
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Main Gun Minimum Effective Range 

The minimum range at which the frigate’s main gun may shoot at. This factor subsumes 
two constraints. First, it is the physically constrained distance at which the main gun can fire 
from its mounting due to barrel minimum elevation. Secondly, it is the range at which a pro¬ 
jectile will be armed and ready to engage. Minimum effective range is varied between 150 and 
350 yards. 

3.3.3 Auxiliary Gun Factors 

All frigates modeled with an auxiliary gun have exactly two or four auxiliary guns. They 
are equally distributed on the port and starboard sides of the frigate model. The auxiliary gun 
has seven variable factors (Table 3.4). Factors modeled by a NOLH design are the auxiliary 
gun’s hit probability, ammunition, firing arc and its maximum effective range. Minimum range, 
inter-firing times and the rounds impact probable lethal range (hit radius) are represented by 
dedicated scale factors modeled by NOLH. In multiple preliminary runs, these factors were 
shown not to be significant predicting damage to the frigate. 

Auxiliary Gun Hit Probability 

This factor describes the single engagement hit probability for a burst of the auxiliary 
gun’s projectiles. It is held constant in between the auxiliary gun’s minimum and maximum 
effective range. The hit probability is varied between 0.01 and 0.98. 

Auxiliary Gun Ammunition 

This factor describes the amount of ammunition for the auxiliary gun as carried by each 
weapon station. The amount of ammunition is varied from 7 to 20 bursts. 

Auxiliary Gun Inter Firing Time 

The gun associated inter firing time. This factor subsumes such factors as reloading, 
target acquisition and aiming. The frigate’s auxiliary gun is modeled shooting between 4 and 
12 aimed bursts per minute. 
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Auxiliary Guns Firing Arc 

The firing arc of the auxiliary guns. This factor describes the physical and superstruc¬ 
ture constraints of the weapons placement aboard the ship. The firing arc has its center on the 
port or starboard side of the frigate, and is divided in equal sections to both sides. Each firing 
arc has a minimum value of 90° and a maximum value of 220°. 

Auxiliary Gun Lethal Radius 

The distance from point of impact around which a round is expected to inflict damage 
to a target. Many different sizes and types of rounds are used in naval warfare. Against SAFC, 
lethal radius is varied between 50 and 100 yards. 

Auxiliary Gun Maximum Effective Range 

The maximum range at which an efficient use of the expended round(s) is to be ex¬ 
pected. Against SAFC, the MER is varied between 300 and 2250 yards. 

Auxiliary Gun Minimum Effective Range 

The minimum range at which the frigate’s auxiliary guns may shoot. This factor sub¬ 
sumes two constraints. First, it is the physically constrained distance at which the auxiliary gun 
can fire from its mounting due to barrel minimum elevation. Secondly, it is the range at which 
a projectile, if active, will be armed and ready to engage. Minimum effective range is varied 
between 50 and 150 yards. 


Item 

min 

max 

units 

varied by 

Hit probability 

0.01 

0.98 

P(hit) 

NOFH 

Ammunition 

7 

20 

bursts 

NOFH 

Inter firing time 

4 

12 

bursts/minute 

scale factor 

Firing arc 

90 

220 

degrees 

NOFH 

Fethal Radius 

50 

100 

yards 

scale factor 

Maximum effective range 

300 

2250 

yards 

NOFH 

Minimum effective range 

50 

150 

yards 

scale factor 


Table 3.4: Auxiliary Gun factor design 
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3.3.4 Small Guns Factors 


If enabled, the frigate is modeled having multiple small guns. The small gun has five 
variable and two fixed factor (Table 3.5). Factors modeled by a NOLH design are the small 
gun’s hit probability ammunition and its maximum effective range. Inter-firing times and the 
round’s impact probable lethal error probable (projectile spray) are represented by dedicated 
scale factors modeled by NOLH. Small gun firing arc and minimum range factors are not var¬ 
ied. In multiple preliminary runs, these factors were shown not to be significant predicting 
damage to the frigate. 

Small Gun Hit Probability 

This factor describes the single engagement hit probability for a burst of the small gun’s 
projectiles. It is held constant in between the small gun’s minimum and maximum effective 
range. The hit probability is varied between 0.01 and 0.99. 

Small Gun Ammunition 

This factor describes the amount of ammunition for the small gun as carried at each 
weapon station. The amount of ammunition is varied from 6 to 20 bursts. 

Small Gun Inter Firing Time 

The small gun associated inter firing time. This factor subsumes such factors as reload¬ 
ing, target acquisition and aiming. The frigate’s small guns are modeled shooting between 5 
and 20 aimed bursts per minute. 

Small Gun Firing Arc 

The firing arc of the small guns. This factor describes the physical superstructure con¬ 
straints of the weapons placement aboard the ship. The firing arc has its center at the bow of the 
frigate, and is covering 360° around the ship, without any variation. 

Small Gun Lethal Radius 

The distance from point of aim around which the bursts projectiles are distributed. A 
circular error probable is assumed for small guns. Against SAFC, lethal radius is varied be¬ 
tween 50 and 100 yards. 
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Item 

min 

max 

units 

varied by 

Hit probability 

0.01 

0.99 

P(hit) 

NOLH 

Ammunition 

6 

20 

bursts 

NOLH 

Inter firing time 

5 

20 

bursts/minute 

scale factor 

Lethal Radius 

50 

100 

yards 

scale factor 

Maximum effective range 

200 

1000 

yards 

NOLH 


Table 3.5: Small Guns factor design 


Small Gun Maximum Effective Range 

The maximum range at which an efficient use of the expended round(s) is to be ex¬ 
pected. Against SAFC, the MER is varied between 200 and 1000 yards. 

Small Gun Minimum Effective Range 

The minimum range at which the frigate’s small guns may shoot at. This factor is fixed 
at 50 yards, the minimum distance allowable. 

3.3.5 CIWS 

If enabled, all frigates are modeled with having exactly one CIWS. The CIWS has eight 
variable factors (Table 3.6). Factors modeled by a NOLH design are the CIWS’s hit probability, 
ammunition, firing arc, bow or aft placement and its maximum effective range. 

Minimum range, inter-firing times and the round’s impact probable lethal error proba¬ 
ble (projectile spray) are represented by dedicated scale factors modeled by NOLH. In multiple 
preliminary runs, these factors were shown not to be significant predicting damage to the frigate. 

CIWS Hit Probability 

This factor describes a single engagement hit probability for a burst of CIWS projec¬ 
tiles. It is held constant in between the CIWS minimum and maximum effective range. The hit 
probability is varied between 0.75 and 0.99. 
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Item 

min 

max 

units 

varied by 

Hit probability 

0.75 

0.99 

P(hit) 

NOLH 

Ammunition 

8 

21 

bursts 

NOLH 

Inter firing time 

4 

8 

bursts/minute 

scale factor 

Firing arc 

150 

360 

degrees 

NOLH 

Lethal Radius 

50 

150 

yards 

scale factor 

Maximum effective range 

750 

3000 

yards 

NOLH 

Minimum effective range 

100 

400 

yards 

scale factor 

Placement 

0 

180 

degree 

NOLH 


Table 3.6: CIWS factor design 


CIWS Ammunition 

This factor describes the number of bursts for the CIWS as carried by the frigate. The 
number of bursts is varied from 8 to 21. 

CIWS Inter Firing Time 

The CIWS associated inter firing time. This factor subsumes such factors as reloading, 
target acquisition and aiming, as well as projectile flight times. The frigate’s CIWS is modeled 
shooting between 4 and 8 aimed bursts per minute. 

CIWS Firing Arc 

The firing arc of the CIWS. This factor describes the physical superstructure constraints 
of the weapons placement aboard the ship. The firing arc has its center according to the “CIWS 
placement” factor, and is divided in equal sections to both sides of the center. The firing arc has 
a minimum value of 150° and a maximum value of 360°. 

CIWS Lethal Radius 

The distance from point of aim around which a burst’s projectiles are distributed, or 
point of impact around which lethal fragments are distributed. Many different sizes and types 
of rounds are used in naval warfare. Against SAFC, lethal radius is varied between 50 and 150 
yards. 
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CIWS Maximum Effective Range 

The maximum range at which an efficient use of the expended round(s) is to be ex¬ 
pected. Against SAFC, the MER is varied between 750 and 3000 yards. 

CIWS Minimum Effective Range 

The minimum range at which the frigate’s CIWS may shoot at. This factor subsumes 
two constraints. First, it is the physically constrained distance at which the CIWS can fire from 
its mounting due to barrel minimum elevation. Secondly, it is the range at which a projectile, if 
active, will be armed and ready to engage. Minimum effective range is varied between 100 and 
400 yards. 

CIWS Placement 

The placement of the single CIWS aboard the frigate. This factor influences where the 
center of the weapons arc is at. Modeled are either a bow (0°) or a stern (180°) placement of the 
weapon system. 

3.3.6 Helicopter Factors 

Helicopter Hit Points 

As with the frigate, a helicopters complexity is modeled by hit points. The hit points 
represent two concepts. First, they represent the actual sustainable damage of the helicopter’s 
hull. Second, they represent how good the helicopter’s pilot is at evading incoming fire. The 
helicopter’s hitpoints value is varied extensively, and is implemented by an NOFH design. The 
minimum value of sustainable damage is 2 hits, and the maximum is 4 hits. Any damage sus¬ 
tained is not healed (repaired) during a run. 

Helicopter Speed 

The helicopter’s speed is held constant during each run. This factor is varied by a NOFH 
design. The minimum speed modeled is 50.3 knots and the maximum speed is modeled as 189.4 
knots (Table 3.7). 
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Item 

min 

max 

measure 

varied by 

Sustainable Damage 

2 

4 

hits 

NOLH 

Maximum Speed 

50.3 

189.4 

knots 

NOLH 


Table 3.7: Helicopter factor design 


3.3.7 Air to Surface Missile AGM-114 Hellfire 

If a helicopter is modeled, it is modeled using the AGM-114 Hellfire. The AGM-114 
has three variable and three fixed factors (Table 3.8). Factors modeled by a NOLH design are 
the AGM-114’s rounds of ammunition and its maximum effective range. The AGM-114’s firing 
arc, lethal radius and minimum effective range are set at fixed values. The inter-firing time is 
calculated using a dedicated scale factor. In multiple preliminary runs, inter-firing times were 
shown not to be significant predicting damage to the frigate. 

AGM-114 Hellfire Hit Probability 

This factor describes the hit probability for the AGM-114 Hellfire missile. It is held 
constant between the minimum and maximum effective range. The hit probability is varied be¬ 
tween 0.75 and 0.99. 

AGM-114 Hellfire Ammunition 

This factor describes the amount of AGM-114’s carried by the helicopter. The amount 
of ammunition is varied from 2 to 8 rounds. 

AGM-114 Hellfire Inter Firing Time 

The Helicopter weapons operator’s AGM-114 inter firing time. This factor subsumes 
such factors as target acquisition and missile impact time. The AGM-114 is modeled shooting 
between 4 and 12 rounds per minute. 

AGM-114 Hellfire Firing Arc 

The firing arc of the AGM-114 Hellfire. The hellfire may change its trajectory at any 
time, and the helicopter supplies a highly mobile launch platform. Thus, even though the Heli¬ 
copter has to face its target straight forward at all times, the firing arc is fixed at 360°. 
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Item 

min 

max 

units 

varied by 

Hit probability 

0.75 

0.99 

P(hit) 

NOFH 

Ammunition 

2 

8 

rounds 

NOFH 

Inter firing time 

4 

12 

rounds/minute 

scale factor 

Maximum effective range 

2500 

5000 

yards 

NOFH 


Table 3.8: AGM-114 Hellfire factor design 


AGM-114 Hellfire Maximum Effective Range 

The maximum range at which an efficient use of the expended round(s) is to be ex¬ 
pected. Against SAFC, the MER is varied between 2500 and 5000 yards. 

AGM-114 Hellfire Lethal Radius 

The maximum range at which an exploding round may inflict damage to a target. Many 
different sizes and types of rounds are used in naval warfare. Against SAFC, an AGM-114 
lethal radius is set at 50 yards. 

AGM-114 Hellfire Minimum Effective Range 

The minimum range at which the AGM-114 may shoot at. This factor describes the 
range at which the AGM-114 will be armed and ready to engage. Minimum effective range is 
fixed at 550 yards. 

3.4 Noise Factors 

3.4.1 SAFC Setup 

Factors modeled for the SAFC are their hit points, number of attackers and speed. All 
factors are varied using a Hadamard matrix, modeling the minimum and maximum of each 
factor. 

SAFC Hit Points 

The SAFC complexity is modeled by hit points. The hit points represent three concepts. 
First, they represent the actual sustainable damage of the SAFC’s hull. Second, they represent 
how good the SAFC’s pilot is at evading incoming fire. Finally, they represent factors such as 
SAFC size and wave height, influencing the probability to hit the SAFC. The SAFC’s hitpoints 
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Item 

min 

max 

units 

varied by 

Hit points 

I 

4 

hits 

min-max 

Speed 

20.1 

44.8 

knots 

min-max 

Attackers 

4 

8 

amount 

min-max 


Table 3.9: SAFC noise factor design 


value is varied. The minimum value of sustainable damage is 1 hit, and the maximum is 4 hits. 
Any damage sustained is not healed (repaired) during a run. 

SAFC Speed 

The SAFC speed is held constant during each run. The minimum speed modeled is 20.1 
knots and the maximum speed is modeled as 44.8 knots (Table 3.9). 

SAFC Numbers 

The SAFC are attacking the frigate using different swarm sizes. The attack is modeled 
with either four or with eight craft attacking. 

3.4.2 RPG-7 

The SAFC are always equipped with at least one RPG-7 launcher. The RPG-7 launcher 
is modeled with six variable and two fixed factors (Table 3.10). Factors modeled by a Hadamard 
matrix design are the RPG-7’s hit probability, number of rocket propelled grenades (RPG), inter 
firing time, minimum effective range, and the number of RPG-7 launchers used. The maximum 
effective range is varied by combining two Hadamard rows, iterating through four values. The 
firing arc and lethal radius are not varied. 

RPG-7 Hit Probability 

This factor describes the hit probability for the RPG-7’s rocket propelled grenades. It 
is held constant in between the minimum and maximum effective range. The hit probability is 
varied between 0.2 and 0.9. 

RPG-7 Ammunition 

This factor describes the amount of RPG’s as carried by the SAFC. The amount of am¬ 
munition is varied between 2 and 8 rounds. 
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RPG-7 Inter Firing Time 

The RPG-7 operator’s inter firing time. This factor subsumes such factors as reloading, 
target acquisition and grenade flight time. The RPG-7 is modeled shooting between 2 and 6 
rounds per minute. 

RPG-7 Firing Arc 

The firing arc of the RPG-7. The shooter must control the firing arc in order not to 
damage the boat or kill members of the crew. The firing arc is fixed at two 45° arcs on port and 
starboard per launcher. 

RPG-7 Lethal Radius 

The maximum range at which an exploding round may inflict damage to a target. Many 
different types of grenades may be shot with the RPG-7 launcher. Against a frigate and heli¬ 
copter, the RPG-7 lethal radius is set at 50 yards. 

RPG-7 Maximum Effective Range 

The maximum range at which an efficient use and decent flight characteristics of the 
expended round(s) is to be expected. Against frigate and helicopter, the MER is varied between 
400 and 800 yards. 

RPG-7 Minimum Effective Range 

The minimum range at which the RPG-7 may shoot at. This factor describes both the 
range at which the RPG-7 will be armed and ready to engage, as well as the minimum distance 
to which the SAFC will close in. Minimum effective range is varied between 50 and 100 yards. 

RPG-7 Number of Launchers 

The maximum number of launchers modeled aboard each single craft. All craft will 
have the same number of RPG-7 launchers. The number of available projectiles is multiplied 
by the number of launchers. Available launchers are varied between one and two launchers per 
craft. 
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Item 

min 

max 

units 

varied by 

Hit probability 

0.2 

0.9 

P(hit) 

min-max 

Ammunition 

2 

8 

grenades 

min-max 

Inter firing time 

2 

6 

grenades/minute 

min-max 

Maximum effective range 

400 

800 

yards 

four values 

Minimum effective range 

50 

100 

yards 

min-max 

RPG-7 launchers 

1 

2 

amount 

min-max 


Table 3.10: RPG-7 noise factor design 


3.4.3 Handheld Weapon 

The SAFC are always equipped with some handheld weapons, represented as one single 
handheld weapon. The handheld weapon is modeled with four variable and three fixed fac¬ 
tors (Table 3.10). Factors modeled by a Hadamard matrix design are the handheld weapon’s 
hit probability, ammunition, and inter firing time. The maximum effective range is varied by 
combining two Hadamard rows, iterating through four values. The firing arc, lethal range, and 
minimum effective range are not varied. 

Handheld Weapon Hit Probability 

This factor describes the single engagement hit probability for a burst of a handheld 
weapon. It is held constant in between the minimum and maximum effective range. The hit 
probability is varied between 0.3 and 0.9. 

Handheld Weapon Ammunition 

This factor describes the amount of ammunition carried by the SAFC. The amount of 
ammunition is varied between 2 and 12 bursts. 

Handheld Weapon Inter Firing Time 

The handheld weapon operator’s inter firing time. This factor subsumes such factors as 
reloading and target acquisition. The handheld weapon is modeled shooting between 5 and 15 
bursts per minute. 
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Handheld Weapon Firing Arc 

The firing arc of the handheld weapons. The shooter is basically free to fire in any di¬ 
rection. The firing arc is fixed at 360°. 

Handheld Weapon Lethal Radius 

The circular error probable around the aim point of the handheld weapon. Against a 
frigate and helicopter, the handheld weapon’s lethal radius is set at 50 yards. 

Handheld Weapon Maximum Effective Range 

The maximum range at which an efficient use of the expended round(s) is to be ex¬ 
pected. Against frigate and helicopter, the MER is varied in 4 steps from 400 to 900 yards. 

Handheld Weapon Minimum Effective Range 

The minimum range at which the handheld weapon may shoot at. Minimum effective 
range for handheld weapons is set fixed at 50 yards. 


Item 

min 

max 

units 

varied by 

Hit probability 

0.3 

0.9 

P(hit) 

min-max 

Ammunition 

8 

12 

bursts 

min-max 

Inter firing time 

5 

15 

bursts/minute 

min-max 

Maximum effective range 

400 

900 

yards 

four values 


Table 3.11: Handheld Weapon noise factor design 


3.5 Experiment Setup 

In this section, the use of XML and the setup of the actual model runs is described. Then 
the lessons learned of the exploratory and preliminary design runs are described. 

3.5.1 Cluster Runs 

The files describing a complete setup of a single MANA model are called scenario files. 
MANA uses a XML based design for its scenario files setup. XML files are human-readable and 
editable using a standard text-editor. Variables within XML files are ordered in a hierarchical 
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fashion. Each hierarchy may contain zero or more elements and zero or more accompanying at¬ 
tributes. Each element or attribute is addressed by a one-to-one name. Elements and attributes 
can have values assigned to them. These values can be assigned externally using the corre¬ 
sponding element/attribute identifier. 

The first step in designing the study is to complete a working MANA scenario file con¬ 
taining all behavior, state changes and fixed value parameters as described in the DoE. The 
design of the base case is not trivial, as some of the model’s desired behaviors are not natively 
supported by MANA. Both were related to ROE’s: Eiring only at an attacking SAEC, and only 
firing at SAEC when they cross within a predefined distance. 

Defending only against a single firing SAEC was modeled using the “Stealth” feature. 
By setting stealth to 100 the SAEC are not detectable by the frigate agent until they fire at the 
frigate. Using a state change stealth is set to 0 once they fire. Thus, the SAEC are detected by 
the frigate. 

Attacking SAEC once they cross within the frigate agent’s safe zone is modeled by us¬ 
ing the “Refueling” feature. Once the SAEC cross this distance, they will change the frigate’s 
state to “being refueled,” triggering a defensive action by the frigate agent. This will also set 
stealth to 0, if applicable. 

Next, a special XME file describing all factor’s identifiers in the order of the columns 
of the M matrix is created. The software Old McData can read this XME file and, using a base 
case of the study setup with the M matrix, is used to control and create threads on the SEED 
Center’s 38 core cluster. In addition to an ID and the study’s author personal information, the 
Old McData XME file also contains the number of desired replications for each design point. 

A typical run of the final model on one the SEED Center’s cores takes 32^4 seconds. 
The exploratory study’s runtime was 58l7 seconds. This improvement was achieved by amend¬ 
ing a stop condition according to the number of agents present (according to the design point) 
to the Old McData file."*^ 

“^'For example, in a design point with 4 SAFC, a frigate and a helicopter, the stop condition was a) Frigate 
reaches goal (crossed the whole screen); b) 4 dead red (SAFC) agents; c) 2 dead blue (Frigate + Helicopter) 
agents. 
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3.5.2 Exploratory Design 

The exploratory design was run as part of the International Data Farming Workshop 
(IDFW) 18 in Monterey, California. As runtime was not yet known, only two replications over 
the design space were performed. Two analyses were conducted. Results from these runs are 
described in (Abel et ah, 2009). 

During the design of the C matrix, a conversion error was made converting real world 
speed of the SAFC and frigate agents into MANA units. This error was discovered by visual 
analysis of both the original run’s unabridged result data and visual replay of certain interesting 
runs within this design space. Speeds were four times faster than specified in the exploratory 
DoE. This mistake invalidates most of this model’s results, and may be considered a Type ITT 
error (Sanchez, 2006). Mitroff and Featheringham (1976) describe a type III error as “the error 
[...] of having solved the wrong problem”. 

Insight was gained insofar as by using statistical analysis the helicopter was by far the 
most significant parameter, followed by frigate hitpoints and auxiliary hit probability. All two- 
way interactions also contained the helicopter. The generated model explains 86.4 percent of 
observed variability using five parameters and three interactions. Only hit probabilities and the 
frigate’s damage sustainability were of interest. Mean frigates lost was very much higher than 
expected in these runs. 

Since these runs were rendered invalid, the same DoE was used for the next setup, albeit 
correcting the conversion errors. Additionally, the scenario’s maximum runtime was lengthened 
to 12000 time steps, as some runs were not finished after 8000 time steps. 

Visual inspection was used to validate the overall design after the data transformation 
fix was applied. A total of 7 design points were run visually to deduce a realistic behavior as 
thought to be within an expected response surface. Some minor modifications were applied to 
the SAEC starting and target areas, such as to result in a higher probability of intercept of SAEC 
and frigate. 
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Figure 3.2: The final setup of the MANA simulation map, with 8 SAFC and a singie frigate agent shown in their 
starting positions. 

3.5.3 Preliminary Design 

The preliminary design was run over 10 replications of 2580 design points. The average 
run length to simulate a single design point was 62.5^5 seconds, a slight increase over the ex¬ 
ploratory design’s runtime of 58l7 seconds. This increase was due to an increased runtime to 
12000 time steps without adjusting the model’s stop conditions. 

A quick analysis of the data showed a huge percentage of runs with no red-blue inter¬ 
actions at all. Only 8000 out of the 25800 runs resulted in any units killed. Further statistical 
analysis yielded a model explaining 83 percent of observed variability with seven parameters 
and three interaction terms. The helicopter did not show up as significant within this analysis. 
Contrary to the predictions made when designing the DoE, some of the lockstepped factors 
were shown to be significant. 
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The preliminary design’s analysis showed a very important factor when dealing with 
SAFC. If the frigate is at least 7 knots faster than the SAFC, these will not be able to intercept it 
without pre-arranged tactics. As this fact is established and this thesis focuses on the effective¬ 
ness of the frigate’s defensive armarment, there is no need for further research in this matter. As 
a result of these insights the minimum SAFC speed for the improved DoE is increased to 20.1 
knots. 


An interesting result was the presence of the “ROE engagement range” being very sig¬ 
nificant using statistical methods. Most important was a range >950 yards — the maximum 
range for the RPG 7. ROE range was varied between 200 and 2000 yards. The RPG 7 is the 
main weapon of the SAEC, and attacking them before they can use this weapon should prove to 
be significant. Therefore the ROE engagement range >950 yards being a significant parameter 
is a validation of the models behavior. 

Eurther analysis generated a model that is partially relying on factors that are lock- 
stepped. To understand which factors are actually responsible for the effect, these lockstepped 
factors need to be individually analyzed. Eor the preliminary model, two independent multi¬ 
plication factors were used. The factors showing in the analysis were all part of one of these 
factors, varying the weapons ammunition, the auxiliary gun inter-firing time and the weapon’s 
firing arcs. These factors will now be varied individually. This design decision necessitates the 
use of the 29 factor NOEH spreadsheet. 

3.5.4 Final Design 

The final design is run over 12 replications of 5140 design points. The model contains 
72 factors, of which 56 are controlling parameters possibly significant on the runs result. Im¬ 
portant changes to the last model are the additional independently NOEH varied factors in the 
DoE. Also, the SAEC have a much higher minimum speed in these runs. The placement of 
the CIWS is now randomized, to see whether there is a difference in between a bow and an aft 
placement. Einally the “ROE engagement range,” the distance the frigate will begin shooting at 
approaching SAEC, is adjusted to 200-1000 yards, as the previous design was considered too 
far. 
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To reduce runtime, stop conditions were introduced to the model based on the number 
of blue and red agents killed. Using this method the average runtime was greatly reduced. In 
total the cluster worked 14 hours and 16 minutes for the computation of the 61680 runs. 
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CHAPTER 4: 
Data Analysis 


The previous three chapters focused on explaining the initial idea, development and im¬ 
plementation of the simulation experimental design. With the final run finished, a large output 
file awaits analysis. 

In this chapter, the output file and the programs and tools used for analysis are described 
briefly. This is followed by a thorough analysis in relation to the MoE, frigate’s survival. Fi¬ 
nally, the analysis is repeated for only those scenarios without helicopter presence. 

4.1 Tools and Methods for Data Analysis 

Before any analysis may be performed, the raw output data needs to be visually in¬ 
spected and cleaned of non-runtime specific data. Non-numeric and indicator variables need to 
be translated, and design point iterations need to be summarized by producing estimates of loss, 
as described in the previous chapter. 

4.1.1 Preparing the Data File 

The resulting data stream generated by the models is a Comma Separated Values (CSV) 
file. It contains all input factor levels for each run, followed by that run’s control and result 
values. For this thesis, the resulting file contains 94 columns and 65524 rows. 

Data Inspection 

First, a visual inspection of the output file is performed. It is important to do this anal¬ 
ysis in order to assess whether information contained in the raw data might be lost when the 
data are summarized. For example, in this study it was discovered that at certain speed ratios, 
the frigate would simply outrun the SAFC, resulting in no frigates lost. This was an important 
finding that is not part of the summarized data. When it was clear that speed has this effect, the 
input speed ranges were reassessed and a new set of model runs were started. 

A visual inspection also includes a quick graphical analysis of data rows, using for ex¬ 
ample distributions and 3D scatter-plots. 
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Data Adaptation 

Translation is done so that statistics tools can correctly interpret the data. In this study 
MANA uses a binary variable to indicate whether weapons are enabled or disabled. This out¬ 
come must be translated to an indicator variable^^ for both summarizing and analysis. Addition¬ 
ally, all values dependent on the binary value are then copied in a new column and multiplied 
by the indicator value. Using this technique, the significance of the binary variables is retained. 

Data Summary 

Summarizing the iterations collapses each iteration of the noise factors and replications 
thereof over the signal space design point, producing an estimated mean and standard deviation. 
This significantly reduces the result set. A part of this activity is the stripping of any data not 
part of the engagement, such as for example data that was only needed to control the model. 

4.1.2 Loss Function 

It was decided to use a squared-error loss function where r is the target value:"^^ 
let 

k = {X.-rf 

= (X,-/i)2 + (r-/i)2-2(Xi-/r)(r-/i) 

then 

E[k] = E[{Xi- + {t - - ^l)] 

= E[{X, - /i)2] + (r - /i)2 - 2(r - ^^)E[{X, - /r)] 

= + (r — /i)^ — 2(r — /i)0 

= + (r - iiY 

= -f /i^ (when r = 0) 

indicator variable is one that can only be “0” or “1.” 

“^^Where the target value r is 0 for X = frigates lost, li is the actual loss for state i, and E[li] is the loss predicted 
for state i. 
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As the Values of and jj^ are not known, they are estimated with and x'^. Thus the 
loss function is calculated by averaging each design point over the design point replications and 
noise factors while generating the sample mean (x) and variance (s^) for the number of frigates 
killed, resulting in the original signal design matrix. Using these statistics a loss function is 
calculated: The loss function has a target value of zero frigates lost, since zero frigates 

killed is optimal. 

The combination of mean squared and variance penalizes being far off the target value 
T of zero in expectation, having highly unstable results as determined by the variability cr^, or 
both. A low loss using this formula is achieved by consistently being at or around zero frigates 
lost. 


The benefit of using a loss function is that it captures the impact of factors outside of the 
experimenters control, while remaining focused on the impact of decisions which are under the 
experimenters control. 

4.1.3 Regression Analysis 

A regression analysis tries to estimate coefficients for some function of the input set¬ 
tings, X, so as to yield the “best” fit for the set of observed outcomes, Y (Goodnight and Sail, 
2008). For a regression as used in this study, the resulting function is comprised of linear and 
non-linear terms. 

Least Squares Regression 

The fundamentals of least-squares analysis were developed by Carl Friedrich Gauss in 
179544 jQ calculations for the movement of cosmic bodies (Dunnington et ah, 2004). 

Stepwise Regression 

Stepwise regression uses regression models in which the selection of predictive variables 
for a least squares model is performed by an automatic procedure which seeks to maximize the 
increase in R^, the proportion of explained variation in the model, at each stage. 


"^"^Gauss was able to prove the statistical characteristics of the method in 1809, assuming normally distributed 
errors. 
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Due to the large number of faetors that are explored, the number of potential terms in 
the model grows rapidly. Therefore only two-way interaetions and polynomials up to degree 2 
are examined. 

The signifieanee-thresholds to enter and leave the model are set at P = 0.001 and 
P = 0.002, respeetively. The variables seleeted for inelusion are the input faetors for least 
squares analysis. 

Due to the large number of observations, a statistieal signifieanee level of less than 
P = 0.0001 is observed for many variables. To be able to differentiate in between statistieal 
signifieanee and the goal of gaining insight, the regression model is restrieted to only those fae¬ 
tors with an |T| > 5 (Figure 4.1). 

4.1.4 Partition Tree 

A partition tree finds a set of euts or groupings of X values that best prediet the ob¬ 
served Y value. It does this by exhaustively searehing all possible euts in the input data. These 
partitions of the data are done reeursively, forming a tree of deeision rules until the desired fit is 
reaehed (Goodnight and Sail, 2008). 

In short, a partition tree is an exploratory model that is a visual tool to reveal strueture 
within a data set (Goodnight and Sail, 2008). It is non-parametrie in that it does not assume 
a partieular funetional relationship between inputs and outputs. It is very good at identifying 
abrupt transitions. 

4.2 Overall Analysis 

For the overall analysis, the aforementioned teehniques are used to build models pre¬ 
dieting the system’s loss in relation to input faetors. The analyzed data set eontains 1028 rows 
of data with 68 independent input faetors and one MoE eaeh. The 1028 rows are a eombination 
of the third preliminary and the final data run. 
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I Sorted Parameter Estimat^ 


Terin_ 

FrigateHP 

SmallGunHitP 

AuxGunhitProbability 

HelicopterAirborne 

MainGurHitProbability 

(SmallGunHitP-5000.02r(SmallGunEnable-0 75097) 

SAFcStealth 

(AuxGunbitProbability-4950.01)*(AuxGunEnable-0.62646) 
(SmallGunEnable-0.75097)*(AuxGunArc-155.004) 

AuxGunArc 

(HelicopterAirbome-0.61089)’(SAFcStealth-50) 

SmallGunAmmo 

(FrigateHP-22.5019)*(HelicopterAirbome-0.61089) 

AuxGunEnable 

(AuxGunProbablelnterFiringTime-9.22763)*(AuxGunEnable-0.62646) 
(MainGunHitProbability-4500.12)*(ROE_AutoFiringRange(RefuelRange)-12.0311) 
AuxGunProbablelnterFirlngTime 
ROE_AutoFiringRange(RefuelRange) 

(SmallGunAmmo-9.36965)’(SmallGuti Ammo-9.36965) 
(MainGunHitProbability-4500.12)*(FrigateHP-22.5019) 

(FrigateHP-22.5019)*(FrigateHP-22.5019) 

(AuxGunAmmo-8.35019)*(AuxGunMaxRg-25.5019) 
(SmallGunAmmo-9.36965)*(AuxGunhitProbability-4950.01) 

(FrigateHP-22.5019)*(ROE_AutoFiringRange(RefuelRange)-12.0311) 

(CWISAmmo-5.55642)*(MainGunHitProbability-4500.12) 

(MainGunHitProbability-4500.12)nAuxGunhitProbability-4950.01) 

SmallGunMaxRg 

AuxGunMaxRg 

(AuxGunAmmo-8.35019)*(CWISEnable-0.37743) 

(SmallGunMaxRg-12.0311)''(HelicopterAirbome-0.61089) 

(SmallGunMaxRg-12.0311)*(SmallGunErable-0.75097) 
(SmailGunHitP-5000.02)*(SAFcStealth-50) 
(CWISAmmo-5.55642)*(SmallGunHitP-5000.02) 
(MGMaxRg-75.0039)*(AuxGunMaxRg-25.5019) 

CWISEnable 

HellFireProbableInterFiringTime 
(HellFireAmmo-5.00389)*(HelicopterAirborne-0.61089) 
(MainGunProbableKillRad-2.11234)*(CWISProbablelnterFiringTime-6.2179) 
(AuxGunAmmo-8.35019)*(SmallGunEnable-0.75097) 
(CWISAmmo-5.55642)*(MainGunProbablelnterFiringTime-4.2179) 
(AuxGunhitProbability-4950.01)’(FrigateSpeed-13.5019) 

SmallGun ProbableKillRadius 


Estiiiuite 

Std Error 

-0.008977 

0.000306 

-1.322e-5 

4.762e-7 

-1.148e-5 

5.028e-7 

-0.120274 

0.005883 

-1.03e-5 

6.024e-7 

-1.751e-5 

1.328-6 

0.0003377 

2.555e-5 

-1.318e-5 

0.000001 

0.0010589 

0.000084 

-0.000468 

3.7198-5 

-0.000611 

5.2478-5 

-0.007501 

0.000675 

0.0069869 

0.000649 

-0.094092 

0.008978 

-0.00902 

0.000924 

-1.25e-6 

1.304e-7 

-0.003846 

0.000405 

-0.002631 

0.00029 

0.0008052 

0.000092 

1.0511e-6 

1.2398-7 

0.0006913 

8.279e-5 

-0.000133 

1.6198-5 

6.9478e-7 

8.6128-8 

0.0005977 

0.000075 

6.6734e-7 

8.739e-8 

1.7433e-9 

2.368-10 

-0.002162 

0.0003 

-0.000848 

0.00012 

0.0033217 

0.000476 

0.0042807 

0.000621 

-0.005658 

0.000825 

-5.906e-8 

9.0028-9 

4.8887e-7 

7.735e-8 

0.000052 

8.509e-6 

-0.055393 

0.009618 

-0.007088 

0.001249 

-0.009195 

0.001628 

0.0114547 

0.002066 

0.0055191 

0.001 

0.0009654 

0.000176 

8.2307e-7 

1.5138-7 

0.0108878 

0.00204 


t Riitio 


-29.30 

-27.77 

-22.83 

-20.45 

- 17.10 

-13.27 

13.22 

-13.08 

12.61 

-12.57 

-11.65 

- 11.12 

10.76 

-10.48 

-9.76 

-9.58 

-9.49 

-9.09 

8.75 

8.48 
8.35 

- 8.22 

8.07 

7.97 
7.64 
7.39 

-7.21 

-7.10 

6.97 
6.90 

- 6.86 

-6.56 

6.32 

6.11 

-5.76 

-5.68 

-5.65 

5.54 

5.52 

5.48 
5.44 
5.34 
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Figure 4.1: Model Parameter Estimates Overview. 


4.3 Data Distribution 

A first insight on the data was gained by graphing the distribution of loss with an ob¬ 
served number of = 1028 data points. 

On first sight, loss seems to be distributed exponentially, with most data showing no loss 
(Figure 4.2). Maximum loss is at 0.6. The distribution’s mean, upper and lower quartiles are 
0.155^0.127, 0.226 and 0.059, respeetively. There is a tail of outliers, starting above a loss of 
0.45, representing 2.5 percent of all data on loss. 
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Figure 4.2: Distribution of ioss. 

4.4 Significant Factors using Partition Trees 

Partition tree analysis on loss was not eontinued beyond ten splits as further splits did 
not eontribute signifieantly to the models’ predietive power. The resulting model explains 49.6 
percent of variance (Figure 4.3). 

Partitioning suggests that small gun inter-engagement time of at least 12 aimed bursts 
per minute contributes most significantly to a high mission survivability. Further important fac¬ 
tors associated with low loss are the CIWS and the auxiliary guns being able to fire at least 
ten aimed bursts before needing to reload. The analysis also suggests the frigates being able to 
sustain at least 21 hits and a small gun hit probability > 30 percent are important factors. 

If the CIWS can shoot less than ten aimed bursts, an auxiliary gun’s firing arc of at least 
191° and a helicopter with at least six Hellfire missiles is significant. Therefore the Hellfire 
missile in combination with a very large auxiliary gun firing arc"^^ at least partially substitute for 
low inter firing times. 

^^Usually the maximum auxiliary gun firing arc is just below 180°. 
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Figure 4.3: Partition tree dispiaying an of 49.6 percent using 10 splits. Notice the reoccurring factors for Frigate 
Survivability (FrigateFIP, marked). 


If the small gun inter-engagement time is less than 12 engagements per minute, the par¬ 
tition tree suggests that if the frigate is able to sustain the equivalent of 16 hits and has a CIWS 
magazine eapaeity of at least eight aimed bursts, loss is signifieantly lowered. 

Besides the damage sustainability of the frigate, weapons that are preeise or those that 
sport a low inter-engagement time show up as being signifieant using partition trees. These are 
the small-ealiber weapons. The helieopter has only restrieted power as it is only signifieant if 
other eonditions do not apply. Also, the main gun seems not to be signifieant at all within this 
model. 


4.5 Significant Factors using Least Squares Regression 

In eontrast to the partition tree, least squares regression ean also take into aeeount higher 
order terms sueh as interactions and polynomials. Due to memory constrains, for this analysis 
no terms higher than second order are modeled. 

65 factors and factor interactions are identified by the regression model, explaining an 
adjusted 89.62 percent of the variance in loss within the model (Figure 4.4). In order to assess 
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Figure 4.4: Summary of Fit for the Least Squares Regression Anaiysis. 


practical significance in relation to response varianee, as a visual tool a predietion profiler is 
employed. 


The aetual by predieted plot shows a very good distribution of data points. There are no 
obvious outliers, and the shape is not abnormal (Figure 4.5). 


4.5.1 Frigate Speed 

The seeond preliminary study eondueted showed the signifieanee of the frigate’s speed. 
If speed is high enough, the SAFC will have no ehanee to entangle the frigate in a fight. The 
frigate ean thus outrun the engagement, whieh is the best solution in light of the researeh ques¬ 
tion. 


4.5.2 Frigate Survivability 

Statistieally the frigate’s survivability is an important measure (Figure 4.6). It showed 
up as the most important measure in the regression. When eonsidering all faetors, it is important 
to understand that within this study the frigate’s eapability to sustain damage showed diminish¬ 
ing returns. Being able to sustain more than 26 hits did not ehange the resulting survivability of 
the frigate in a signifieant way. As this was an interesting result, a total of 10 design points with 
a high survivability were eonveniently ehosen for visual inspeetion. Changing the simulation 
seed at random to alter the environment, a total of 100 observations were made. As was pre- 
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Figure 4.5: Actual by Predicted Plot using Least Squares Regression Analysis. 

dieted by the partition tree, a frigate that can sustain more than about 25 hits will either destroy 
all attacking SAFC or simply sustain the SAFC attacks until those run out of ammunition. In 
either case, the result is a very low loss. 

This result is also a confirmation and an insight of the model’s predictive power. Hughes 
salvo equations (Hughes, 1995) indicate that the factor of “staying power” is one of the largest 
contributions to a ship’s defense. 
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Figure 4.6: Prediction profiierfor Frigate Survivability (FrigateFIP). 



Figure 4.7: The range at which a contact is deemed hostile has low impact on predicted loss. 

4.5.3 Tactics and Procedures 

Firing range (beginning to shoot), as well as shooting at all small boats vs. shooting 
at only the attaeker, are statistieally significant in the regression. They do not make a visual 
difference in the prediction profiler, though (Figure 4.7). When considering the set-up of the 
simulation, this result is to be expected. The SAFC start off at random locations and with the 
same probability to move in a given turn. On average they will therefore have the same speed, 
and not reach the frigate at the same time. In the case of the frigate attacking only a single 
SAFC, all of the frigate’s applicable weapons fire at that one target. This increases the chance 
of killing the SAFC within a short time, and will happen for each closing SAFC. 

On the other hand, when the frigate attacks all targets at once, it has to distribute a lim¬ 
ited amount of weapons on a potentially larger number of attackers. Thus, overall each of the 
attacking SAFC will be within its weapon range for a longer time and potentially inflict a higher 
amount of damage. But, as the MOE was not number of hits received, very often the frigate 
will simply survive these longer attacks and thus, even though the difference is significant, in 
this simulation it does not make a difference. 
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4.5.4 Main Gun 


The main gun is the only weapon being modeled in all runs. Contrary to the first two 
preliminary run’s results and the partition tree, using regression several of the main guns fae- 
tors showed up as being statistieally signifieant: ammunition, hit probability, inter-engagement 
time, range and firing are. Out of these faetors, using the predietion profiler, hit probability and 
the amount of ammunition available make a visible differenee. 



o o o o 

(N ^ to CO 

Main Gun P(Hit) 


Figure 4.8: Prediction profiier for Main Gun Hit Probability. 

4.5.5 Auxiliary Guns 

Enabling auxiliary guns makes a signifieant differenee in explaining model result vari¬ 
ability, as does the gun’s firing are (Figure 4.9) and ammunition amount (Figure 4.10). Auxiliary 
gun hit probability is the third most important faetor within the regression model. 



Figure 4.9: Prediction profiler for the auxiliary Gun firing arc width. 
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Figure 4.10: Prediction profiler for the amount of ammunition on the auxiliary gun. 

In relation to explaining the model variability, the profiler shows that hit probability, the 
gun’s maximum range and projeetile dispersion (lethal radius) are not signifieant effeets (Fig¬ 
ure 4.11). 

Within this simulation, maximum range was not expeeted to have an effeet and eould 
be seen as a “eontrol” if the statistieal tools over-fit. The engagements as determined by the 
restraints within the simulation’s parameters are all well within eaeh of the modeled weapon’s 
maximum range. It is also interesting to note that the firing are is so mueh more important than 
the hit probability. This seems to indieate that it is better to engage a SAFC even with a low hit 
probability than not to engage it at all. 
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Figure 4.11: Prediction profiler shows no significant changes In survivability for auxiliary gun’s hit probability, lethal 
radius or maximum range. 

4.5.6 Small Guns 

Small guns signifieantly influenee the model’s variability. Espeeially the amount of 
available ammunition (Figure 4.13) and inter firing times (Figure 4.12) have great influenee on 
the variability of the model aeeording to the predietion profiler. 
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Figure 4.12: Profile of small guns inter firing times on model predicted loss. 

Enabling the small gun showed some interesting results within the predietion profiler. 
The small guns presenee or absence will both increase or decrease expected loss. If expected 
loss is low, enabling the small guns will increase loss, while the small guns will decrease overall 
loss if expected loss is high. A convenient sample of 10 design points that resulted in both high 
and low loss were selected and the simulation’s raw result sets visually inspected. For half the 
samples the small gun was enabled, and for the other half disabled. Inspection indicates that 
activating small guns in low expected loss situations only minimally lowers mean missions lost 
while variance is much increased. In high loss situations, variance is not very much affected but 
mean missions lost is marginally lowered. 

Increasing the number of aimed bursts fired per minute does increase expected loss. The 
expected result is that increasing the number of shots fired per minute, regardless of P(hit), re¬ 
sults in more SAFC hit and thus in a lower loss. Visual inspection both of the simulation’s runs, 
as well as raw data, did not help to explain this result. This is either suggesting volatility in the 
results, some kind of interaction that the employed statistical methods do not find, or it may be 
a modeling artifact. 

The effect the amount of ammunition available to the small guns has on the model di¬ 
minishes the more ammunition is supplied. This effect actually reverses at around 14 aimed 
bursts shot, increasing the loss. This effect is possibly related to the rise in loss that is observed 
when aimed shots per minute are increased. Alternatively, it may be because a quadratic fit is 
not the best choice to capture the non-linear response. 
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Figure 4.13: The prediction profiler shows diminishing returns on amounts of ammunition for small guns. 

4.5.7 CIWS 

In explaining the model, the CIWS do not make a signifieant impaet. The profiler shows 
enabling CIWS having some influenee on expected loss, but the factor is not part of the re¬ 
gression model. This does not indicate that the CIWS really has no effect, but shows that the 
CrWS’ effect as set up in the simulation is not as significant relative to the other factors. The 
simulation models highly sophisticated weapon systems, on par with the CIWS on speed and 
hit probability. Therefore the CIWS not being highly significant is acceptable. 

4.5.8 Helicopter 

In the regression model, enabling the helicopter is the most significant factor following 
frigate survivability (Figure 4.14). Additionally, the helicopter has a significant influence ex¬ 
plaining variability within the model. The number of hellfire missiles, or their hit probability, is 
rather insignificant. 
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Figure 4.14: Flaving a helicopter airborne predicts lower loss. 


78 

















4.6 Significant Factors - Effect Interactions 

While it is important to understand which factors have an important influence on the 
outcome of a model, and thus give some insight to the system being modeled, it is not always 
possible to actually integrate all factors. It is of great interest to understand what factors allow 
trade offs or depend on one another. Therefore the analysis of interactions is very important 
when trying to understand a system. 

The regression model determines that there are many proposed significant interactions 
in between factors. These are composed of merely technical interactions, due to the setup of the 
model, and interactions that allow some insight in the relationships between single factors. 

Technical interactions are observed between the indicator variables of enabling the small 
guns, auxiliary guns and helicopter. For the gun systems, the interaction factor is hit probability, 
while for the helicopter it is amount of hellfire ammunition. These factors are dependent based 
on the design of the models run file. 

The presentation of the interactions using a certain section header is for arrangement 
purposes only. Each interaction could also be presented using the other interaction term. The 
insights will also be true if they are reversed. 

Interactions are included in the model only if their respective regression resulting F-ratio 
is greater than 30. 

For each interaction, first the interactions are pictured. Then, each interactions insights 
are discussed. 

4.6.1 Interactions Concerning Frigate Survivability 

There is an interaction between the survivability of the frigate and having a helicopter, 
ROE engagement range and the main gun’s hit probability. 

The model suggests that survivability needs to be higher, if no helicopter is present. The 
helicopter both diverts and reduces the number of SAFC. It diverts the SAFC from the frigate 
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as the SAFC choose whether to attack the frigate or the helicopter. The simulation assigns the 
frigate a higher chance of aggregating SAFC then the helicopter. SAFC can either attack the 
frigate or the helicopter. Also, the helicopter fired Hellfire missiles have a high chance to reduce 
the number of attacking SAFC. 

The lower the ROE engagement range, the higher the frigate’s survivability needs to be 
for a low loss, as more SAFC will be able to discharge weapons before being neutralized. Hit 
probabilities are step functions, so distance only matters for being able to hit or not to hit. If 
the SAFC are close to the frigate, the frigates large- and medium caliber weapon systems have 
minimum effective ranges, thus reducing the number of weapon systems being able to target 
the SAFC. This increases the expected time before a single SAFC is neutralized, increasing the 
time it can fire on the frigate and thus increasing the number of hits the frigate will receive. 

The same reasoning is true for a lower main gun hit probability necessitating a higher 
survivability for the frigate, as the SAFC have a higher probability to close in on the frigate 
before being neutralized. The longer a single SAFC survives, the longer the timespan to fire its 
weapon systems, resulting in a a higher number of expected hits on the frigate. 

4.6.2 Interactions Concerning the Main Gun 

There is an interaction between the main gun hit probability and ROE engagement range 
and CIWS amount of ammunition. There is an interaction between the main gun projectile kill 
radius and the CIWS inter-engagement time. 

A high hit probability of the main gun offsets a reduced ROE engagement range as the 
main gun will reduce the number of SAEC in a shorter amount of time. This is only true while 
the minimum ROE engagement range is still farther than the main gun’s minimum effective 
range, a subtlety that eludes this model. Similarly, a higher hit probability of the main gun 
allows a lower amount of ammunition on the CIWS with the same reasoning. 

If the main gun has a larger projectile kill radius, a longer inter-engagement time for 
the CIWS will achieve the same predicted loss. A larger kill radius will inflict damage on a 
SAEC even if the projectile does not directly hit. This, for all practical purposes, significantly 
increases the hit probability of the main gun, such as this is a subset of the above interaction. 
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4.6.3 Interactions Concerning the Auxiliary Gun 

There is an interaction between the auxiliary gun’s hit probability and main gun hit prob¬ 
ability, frigate speed and small gun’s ammunition. There is an interaction between the amount 
of ammunition available to the auxiliary gun and enabling the CIWS, enabling small guns and 
overall range of the auxiliary guns. There is an interaction between the auxiliary gun’s firing 
arcs and the tactic to attack only definitely hostile SAFC versus all approaching SAFC. 

A higher auxiliary gun hit probability allows for a lower main gun hit probability as the 
same number of auxiliary gun engagements disable a higher number of SAFC. This is also the 
case for the amount of ammunition available to the small guns, which may be lower if the aux¬ 
iliary gun hit probability is increased. A higher frigate speed allows for a lower hit probability 
for the auxiliary guns, as the engagement is prolonged before the SAFC can close in, allowing 
more auxiliary gun engagements. 

Increasing the auxiliary guns’ ammunition significantly interacts with both enabling the 
CIWS and the small guns. While overall loss is lower for enabling both CIWS and small guns, 
a higher amount of ammunition allows more engagements, thus at least partially offsetting the 
higher loss expected by not having these additional assets. Loss is decreased by an interaction 
between high ammunition amounts and high maximum effective range. But these interactions 
are only possible when either increasing the ROE engagement distance, or when classifying all 
approaching SAFC as hostile upon first attack. This allows the auxiliary gun to engage single 
targets for a longer time while they are closing in, thus increasing the probability of neutralizing 
the approaching craft, which in turn reduces loss. The same explanation is true for the auxiliary 
gun’s weapon arc, as a larger weapons arc allows for a higher probability to be able to engage 
approaching SAFC, but only if they are classified as being hostile. 

4.6.4 Interactions Concerning the Helicopter 

There is an interaction between employing a helicopter and attacking only definitely 
hostile SAFC versus all approaching SAFC and a CIWS being present. 
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If only SAFC shooting at the frigate or entering the ROE engagement distance are en¬ 
gaged, the model suggests the use of a helicopter may decrease loss. The helicopter employs a 
high probability of hit weapon, the Hellfire missile. Additionally, the helicopter is highly agile 
and is stationed where it might be able to attack otherwise inaccessible SAFC. The combina¬ 
tion of all of these factors make it a great asset when only single, definitely hostile SAFC are 
engaged. 

With the same reasoning, the use of an airborne helicopter is similar to employing a 
CIWS system. Both systems are high P(hit) systems, are agile but have a comparatively long 
inter-engagement time. Of course, a helicopter has the drawback of only limited availability. 

4.7 Significant Factors when not Employing a Helicopter 

As mentioned in both Chapter 1 and 2, the use of a helicopter during a given operation 
is limited. To determine the impact of not using a helicopter during an engagement, the analysis 
is repeated including the 400 design points in which the helicopter is not employed. 

4.7.1 Influential Factors using a Partition Tree 

Partition tree analysis on loss was not continued beyond seven splits as further splits did 
not contribute significantly to the models predictive power. The resulting model explains 52.2 
percent of variance. 

Partitioning suggests that small gun single engagement hit probability of above P>0.58 
is most significant in predicting low loss configurations. The small gun is the only weapon 
capable of a 360° engagement arc in all design points, has a short minimum engagement range 
and a fairly low inter-engagement time. The combination of these factors allow the small gun, 
if combined with a high hit probability, to dominate as most significant factor. 

If the small gun high P(hit) configuration is paired with the factor auxiliary gun single 
engagement hit probability of above P=0.64, loss is at a predicted minimum of about 0.07 in 
this model. The auxiliary gun also has a fairly large engagement arc and a low inter-engagement 
time. The minimum engagement distance is higher for some auxiliary gun design points, so if 
the SAFC close in on the frigate, the auxiliary gun ceases to be useful. But if the hit probability 
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of the auxiliary gun is high enough, the SAFC will not be able to close in fast enough to bridge 
the engagement zone before being neutralized, resulting in low loss. 

If the auxiliary gun hit probability is below P=0.64, CIWS single engagement hit prob¬ 
ability of above P=0.76 is introduced as an important factor to predict a loss of about 0.12. 
(Figure 4.15). The CIWS has a maximum firing arc of 260 degrees, far less than the aux and 
small guns. It has a higher inter-engagement time, but is also modeled with very much higher 
hit probabilities. Therefore both auxiliary and small guns dominate CIWS as a factor. 



Figure 4.15: Partition matrix dispiaying an of 52.2 percent using 7 spiits. 

Should the small gun single engagement hit probability be below about P=0.58, a frigate’s 
survivability level of above 24, in combination with a low CIWS inter-engagement time, is sig¬ 
nificant in predicting a loss of about 0.09. The frigate needs to be able to withstand more 
damage if the small guns have a lower hit probability. As the CIWS hit probability is set up to 
be high, a shorter inter engagement time results in more SAFC neutralized in the same amount 
of time, thus resulting in a low loss configuration. 

As explained within the model with partial use of a helicopter, if the frigate has only 
moderate survivability levels, above 18 or below 24, the tactic of attacking all approaching 
SAFC without further identification is a significant factor to predict a loss of about 0.22. 
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4.7.2 Influential Factors using Least Squares Regression 


Summary of Fit 
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Figure 4.16: Summary of Fit for the Least Squares Regression Anaiysis of Loss Without Heiicopter. 

27 factors and factor interactions are identified by the regression model, explaining an 
adjusted 91.55 pereent of the varianee in loss within the model (Figure 4.16). The model shows 
fewer faetors to be signifieant as the no-helieopter model determining input parameter list is 
1/3^'^ the length of the with-helieopter analysis. As with the with-helieopter evaluation, in 
order to assess praetical signifieanee in relation to response varianee, a prediction profiler is 
employed as a visual tool. 


The aetual by predieted plot shows a very good distribution of data points. There are no 
obvious outliers, and the shape is not abnormal (Figure 4.17). 


For frigate survivability, single engagement hit probabilities are mueh more signifieant 
when not employing a helieopter. The helieopter, if present, would employ a weapon with a 
high probability of kill, and a low variance in its results. While this weapon is not present, the 
other weapon systems need to make up for both the inereased varianee and the longer time it 
takes to neutralize the approaehing SAFC. Deereasing the inter-engagement times would statis¬ 
tically increase the number of SAFC neutralized within a certain number of timesteps, but may 
also increase variance. Inereasing hit probability will deerease variance and deerease the time 
it takes to neutralize the SAFC, thus, deereasing expeeted loss. 
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Actual by Predicted Plot 



Figure 4.17: The Actual by Predicted Plot for Loss Without Helicopter. 

Assuming all SAFC to be hostile onee the first is determined to be hostile is signifieant 
when not employing a helieopter. The helieopter was shown to be a great asset when attacking 
the SAFC one at a time. Without the helicopter, it is now better to assume all hostile and thus 
gain the maximum utility from especially long-range weapons, such as the main gun, which 
also have a very long minimum engagement distance. 

The prediction profiler shows there are six very influential factors that dominate the 
models predicted loss (Figure 4.19). 

Both small gun ammunition and small gun single engagement hit probability are signifi¬ 
cant, as it allows the small guns to shoot longer and hit more often. Main gun single engagement 
hit probability is also adding a large effect. The main gun is modeled as an area effect weapon, 
so increasing hit probability multiplies the chance of the weapon actually damaging a SAFC, 
even if it does not hit its intended target. Increased survivability, another significant factor, has 
already been shown to decrease loss, as is predicted in this model. The CIWS maximum range 
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Figure 4.18: Model Sorted Parameter Estimates. 



Figure 4.19: Model Sorted Parameter Estimates. 


are predicted to reduce loss. The CIWS is a high P(hit) weapon, so the earlier it can engage a 
target, the more damage it can deal to the SAFC. This damage is also very consistent, as such 
reducing loss. Finally, the auxiliary gun’s single engagement hit probability shows a high influ¬ 
ence on predicted loss. Thus a high hit probability will generally reduce loss. 


The auxiliary gun hit probability as determined by the prediction profiler shows dimin¬ 
ishing returns and a leveling at a P(hit) of about 0.85. This prediction was expected. Increasing 
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a hit probability at a level of P=0.8 and above does not make as much a difference as increasing 
P(hit) from 0.2 (an expected hit in five shots) to 0.5 (an expected hit every two shots). But high 
hit probabilities not only reduce the number of SAFC more efficiently, but also reduce the num¬ 
ber with a low variability in expected results. Inter-trial variability gets smaller the higher the 
P(hit). Thus with a loss function, predictions on P(hit) are expected to show diminishing returns. 

Thus, overall an interesting change is that tactics have some influence on predicted loss. 
Without a helicopter, it is more important to determine all SAFC intentions early, or employ 
ROE that allow us to deduce intentions of a swarm of SAFC through the behavior of any of its 
members. 

When comparing the model partially employing a helicopter to the model not employ¬ 
ing a helicopter, a high overlap in the sets of factors determining loss is observed. Both factor 
weighting and the predicted factor profiles are unique to each model, but the factor effects are 
essentially the same for both scenarios. 
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CHAPTER 5: 

Conclusions And Recommendations 


5.1 Research Summary 

This study investigated how well a typical North Atlantic Treaty Organization (NATO) 
operated Fast Frigate with Helicopter (FFH), built between 1975 and 1990 and operating as a 
lone unit, would perform in a notional Security, Transition and Reconstruction (SSTR) mission 
when attacked by Small, Agile and Fast Craft (SAFC) armed with readily available hand-held 
weapons. A hypothetical defense against different sized swarms of small vessels at different 
speeds in differing environmental conditions was simulated, using the frigate’s survivability as 
the measure of effectiveness. The goal was to determine if current frigate’s survivability needs 
to be increased and if so, how this is to be achieved. The simulation study yielded some inter¬ 
esting insights about the influence of different factors on a frigates mission survivability. 

5.2 Research Questions 

In Chapter 1, the following research questions were posed: 

1. Can a swarm of four to eight small agile fast boats armed with handheld weapons, attack 
and achieve a mission kill on a typical NATO FFH operating in a SSTR mission? 

2. Is the FFH’s mission survival dependent on the sophistication of installed weapons? 

3. What type of weapon or weapons package is best suited for close-in defense against four 
to eight small seaborne attackers? 

4. Would it benefit mission survivability to improve or replace existing weapons of frigates 
performing SSTR missions? 

These questions were based on the premise that the new or updated capabilities could 
at least theoretically be equipped on the average commissioned frigate while not, or only mini¬ 
mally, altering the ship’s hull or its hull structure, respectively. 
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It was found that currently employed NATO frigate’s are vulnerable to an attacking 
swarm of four to eight SAFC. Even though predicted loss is only 0.155^0.127, this number is 
quite high considering the possible impact an actual mission kill might have. 

It was further shown that the sophistication of the employed weapon mix is a significant 
factor in the survival of the frigates. Frigates using automatic, high-probability-of-hit weapon 
systems were significantly more likely to survive a hostile encounter than those using crew- 
served manual-aim weapon systems. 

The observed interactions suggest that besides sophistication, a well balanced mix of 
weapons is key to a low rate of loss. No single sophisticated weapon dominates expected re¬ 
sults. The model also suggests that expected loss is reduced by having enough ammunition 
on the weapons to allow for multiple engagements before a necessary reload. Loss is reduced 
further by using a helicopter employing stand-off, high probability of kill weaponry. 

Based of these findings, recommendations to increase mission survivability include 
modernizing and replacing crew-served with automatic weapon systems, and update large- 
caliber and CIWS weapon systems to enable the engagement of SAFC type targets. Prior 
research suggests integration of modem, automatic and semiautomatic weapon systems in ex¬ 
isting designs is technologically feasible. On the same note, increasing the number of small and 
medium caliber weapon systems while disregarding single engagement hit probabilities^^ has 
no significant effect on the frigate’s survivability. Additionally, the model suggests that due to 
target allocation problems, simply adding random firepower may even decrease overall frigate 
survivability. 

5.3 Further Insights 

In addition to addressing the research questions posed in Chapter 1, some further in¬ 
sights have been gained through this research. This section discusses these insights and the 
impact these findings may have on the frigate’s mission survivability. 


“^^Modeled by drastically increasing inter-firing times. 
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5.3.1 Frigate Speed 

It was observed in the preliminary simulation runs that the higher the frigate’s speed, the 
lower the probability of skirmishes happening. This in turn results in low loss, meaning high 
mission survivability. This concurs with current findings as presented in Chapter 1, that the 
slower a vessel, the higher the probability it will be intercepted by SAFC. Most FFH’s included 
in this study do have high top speed, but high speeds result in very high fuel consumption. Also, 
most frigates have CODOG propulsion. Switching from diesel to gas turbine takes time, and 
switching often decreases turbine lifespan. Both high fuel consumption and turbine life reduc¬ 
tion are undesirable and, thus, the influence of high frigate speed is good to know, but probably 
not very applicable to existing frigate designs. 

5.3.2 SAFC Speed 

It was observed during the preliminary simulation runs that the higher the SAFC speed, 
the more important the helicopter and small caliber, short range weapons are. This is due to the 
fact that the SAFC will swiftly cross the effective range of large caliber weapons, after which 
they may only be targeted by small caliber weapons. The helicopter is significant, as in the 
model there was no restriction on stand-off distance in relation to the frigate. In real life, a 
helicopter will in all probability not shoot at a SAFC alongside a frigate.^^ Nonetheless, if the 
helicopter is positioned well, it might take on a SAFC up to 50 yards off the frigates hull. This 
suggests that with very fast approaching unknown contacts, it is very important to determine 
intent early and swiftly take appropriate measures. 

5.3.3 Helicopter Employment 

It was observed during the simulation runs that employing a helicopter overall increases 
the probability of mission survival. The helicopter is not a dominating factor, though. When 
no helicopter is airborne it is important to determine hostile intentions early, or employ ROE 
that allow to deduce an approaching swarm’s intentions through the action of any one of its 
members. 


“^^Even though the modeled Hellfire missile uses a laser designator, the missile may lose the rather small SAFC 
and lock-on to the frigate instead. According to subject matter experts, no Captain will allow Hellfire Missiles to 
be shot within about 500 yards of its vessel. 
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As observed weapon effects are essentially the same for both scenarios with and without 
helicopter, the model suggests not employing a helicopter may be compensated for by employ¬ 
ing some other kind of sophisticated weapon system. 

5.4 Recommendations 

The insights gained do support the following observations and recommendations on 
frigate tactics, hull and effector design when dealing with small, agile and fast craft: 

1. A frigate should be able to run at high speeds, outmaneuvering the attacker. 

2. Weapons should achieve a high hit probability. 

3. Highly trained human weapon operators achieve only very low P(hit). 

4. Lower weapon’s aimed inter-engagement times. A good example are modern CIWS. 

5. Increase ammunition available on individual weapons, or reduce reloading times. 

6. Employ a helicopter armed with a sophisticated maritime stand-off weapon whenever 
possible. 

7. Large caliber ASW weapons are of no use in SAFC engagements. 

All these suggestions are only valid in relation to a defense against up to eight small 
craft. This research does not suggest that these weapons are also effective against other types 
of threats. While they will probably not weaken power projection, they might not help. 

It is also important to understand that even when all recommendations are put in practice, 
there is no guarantee that this will not result in a mission kill. As Clausewitz says, 

... Absolute, so-called mathematical factors never find a firm basis in military 
calculations ... In the whole range of human activities, war most closely resem¬ 
bles a game of cards... The art of war deals with living and with moral forces. 
Consequently, it cannot attain the absolute, or certainty; it must always leave a 
margin for uncertainty, in the greatest things as much as in the smallest (Clause¬ 
witz and Schlieffen, 1905). 


92 



However, the major benefit of using robust analysis methodology is that they account 
for and mitigate the effects of uncertainty. Usage of these techniques is one of the contributions 
of this thesis. 


5.5 Further Research 

This study may well be expanded to include other MoE’s, threats or higher level of 
detail. The following list surely is by no means comprehensive: 

1. Employ suggested weapon mix in other scenarios and develop further insights. 

2. Use of red teaming to find weaknesses in the improved defenses of the frigate. 

3. Increase the attacking SAEC swarm size to determine limitations in the suggested setup. 

4. Model catastrophic events such as hits in a magazine. 

5. Model effector and system deterioration with damage received. 

6. Model the effect of armoring on survivability. 

7. Include a model of the frigate in sections, reducing the level of abstraction. 

8. Model curves for weapon hit probabilities, sea states and greater variability of attackers 
capabilities. 

9. Develop a set of mathematical equations describing interactions between frigate and 
SAEC. 

10. Modeling partial or full integration of new automatic weapons in a certain frigate’s C2 
system, run a cost-benefit analysis, considering both the probability of attack and the 
cost of a mission kill of a frigate. 

5.6 Impact of Time 

While this study was in the process of being written, some of this study’s recommenda¬ 
tions began being implemented. Defense news magazines, e.g., Jane’s Defense Weekly (Janes 
Defense Weekly, 2009) and Naval Technology (Naval Technology, 2009), report that some 

“^^Considering political issues, evaluate cost of follow-on missions and cost of human/system losses. 
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navies are in the proeess of upgrading some of their weapon systems to use new targeting soft¬ 
ware, automated targeting and traeking modules, and eomputer-eontrolled turrets. 

It is reassuring that the reeommendations made within this study eoineide with the de- 
eisions taken by the defense seetor. 
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Naval Weapon Data Overview 

This section covers the technical details, as far as they are known, of the weapons that 
were evaluated when determining the input data for the models run. The sections include Main 
Guns, Auxiliary Guns, Close In Weapon Systems, Small Guns and Handheld Weapons. 

Main Guns 

OTO Melara 76 mm gun (traditional/compatto/rapid) 

• Effective range: 19000 meters 

• Min range: 300 meters 

• Caliber: 76 mm (3 inch) 

• Rate of Fire: 35 - 120rds/min 

• Magazine Capacity: 80 rounds 

• Traverse: 35 degrees / second 

• Alias: U.S Mark 75 

Bofors 57/70 mm MKII / MKIII 

• Effective range: 14500 meters 

• Min range: 250 meters 

• Caliber: 57 mm 

• Rate of Fire: 60 - 140 rds/min 

• Magazine Capacity: no limit 

• Traverse: 65 degrees / second 
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CADAM Turret / Loire 100mm / MK55 Mod 68 


• Effective range: 12000 meters 

• Min range: 300 meters 

• Caliber: 100 mm 

• Rate of Fire: 30 - 90 rds/min 

• Magazine Capacity: 60 

• Traverse: 40 degrees / second 

OTO Breda 127/54 

• Effective range: 24000 meters 

• Min range: 300 - 800 meters 

• Caliber: 127 mm (5 inch) 

• Rate of Fire: 40 rds/min 

• Magazine Capacity: 22 x 3 (66) 

• Traverse: 40 degrees / second 

OTO Melara 127/64 

• Effective range: 50000 meters 

• Min range: 500 - 800 meters 

• Caliber: 127 mm (5 inch) 

• Rate of Fire: 35 rds/min 

• Magazine Capacity: 22 x 3 

• Traverse: 35 degrees / second 
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114mm Mark 8 Mod 0 / Mod 1 (4.5 inch) (Fischer, 2006) 

• Effective range: 220000 meters 

• Min range: unknown 

• Caliber: 114 mm (4.5 inch) 

• Rate of Fire: 25 rds/min 

• Magazine Capacity: 1 or 18 (Mod 0 / Mod 1) 

• Traverse: 42 degrees / second 

Giat CADAM Turret 

• Caliber: 100 mm 

• Rate of Fire: 25 rds/min 

Auxiliary Guns 

OTO Breda 40/L70 twin 

• Effective range: 125000 meters 

• Min range: 50 meters 

• Caliber: 40 mm 

• Rate of Fire: 300 rds/min 

• Magazine Capacity: 444 

• Alias: Type 64 / FastFourty 

Mauser EADS MEG 30/27 mm 

• Effective range: 3000 / 3500 meters 

• Min range: 25 meters 
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Caliber: 25 / 30 mm 


Rate of Fire: 800 rds/min 
Magazine Capacity: 90 to 105 rounds 
Alias: Marineleichtgeschuetz 

Rheinmetall GDM-08 with MSP 500 

Effective range: 5500 meters 
Min range: unknown 
Caliber: 35 mm 
Rate of Fire: 1000 rds/min 
Magazine Capacity: 1200 rounds 
Alias: Millennium NGS 

Oerlikon Gam/BO 1 

Effective range: 2000 meters 
Min range: unknown 
Caliber: 20 mm 
Rate of Fire: 1000 rds/min 
Magazine Capacity: 200 rounds 

Allied Telesyn DS 30M Automated Small Calibre Gun System 

Effective range: 2000 meters 
Min range: unknown 
Caliber: 20 mm 
Rate of Fire: 200 rds/min 
Magazine Capacity: 150 rounds 
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Rheinmetall RH 202 


• Effective range: 2500 meters 

• Min range: 25 meters 

• Caliber: 20 mm 

• Rate of Fire: 880- lOOOrds/min 

• Magazine Capacity: 30 - 120 rounds 

Close In Weapon Systems 

Mauser Oerlikon MeRoKa 

• Effective range: 3000 meters 

• Min range: 150 meters 

• Caliber: 20 mm 

• Rate of Fire: 1440 rds/min 

• Magazine Capacity: 2160 rounds 

Signaal GAU-8/A 

• Effective range: 1500 meters 

• Min range: 100 meters 

• Caliber: 30 mm 

• Rate of Fire: 4200 rds/min 

• Magazine Capacity: 14 * 90 rounds 

• Alias: Goalkeeper, GE-30 
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GE / GDC MK 15 Mod 2 


• Effective range: 2500 meters 

• Min range: unknown 

• Caliber: 20 mm 

• Rate of Fire: 4500 rds/min 

• Magazine Capacity: 1250 rounds 

• Alias: Phalanx 

Raytheon / Diehl RIM 116 Block 1 HAS 

• Effective range: 7500 meters 

• Min range: 250 meters 

• Caliber: 127 mm (5 inch) 

• Rate of Fire: 30 rds/min 

• Magazine Capacity: 21/11 rounds 

• Alias: Rolling Airframe Missile (RAM), Mk 44 

Small Arms 

Browning M3M 

• Effective range: 2000 meters 

• Min range: 0 to 25 meters 

• Caliber: 12.7 mm (0.5 inch) 

• Rate of Fire: 500 - 1200 rds/min 

• Magazine Capacity: 50 - 200 rounds 

• Alias: Ma Deuce, .50 Browning 
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GE M 134 GAU-2B 


• Effective range: 750 meters 

• Min range: 0 meter 

• Caliber: 7.62 mm (0.308 inch) 

• Rate of Fire: up to 6000 rds/min 

• Magazine Capacity: up to 1500 rounds 

• Alias: Minigun 

OTO Melara 12.7mm HITROLE NT 

• Effective range: 2500 meters 

• Min range: 5 to 25 meters 

• Caliber: 12.7 mm (0.308 inch) 

• Rate of Fire: 500 - 1200 rds/min 

• Magazine Capacity: 110 to 400 rounds 

Handheld Weapons 

RPG - 7 40mm w/ HEAT grenade 

• Max effective range: 350 m 

• Max absolute range: 940 m (4.5 sec flight) 

• Weight loaded: 12 kg 

• Rounds per minute: 2-4 

• Country of origin: USSR 

• Number Produced: 9000000 
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RPK 7.62 mm light machine gun 

• Max effective range: 800 m 

• Weight loaded: 6 kg 

• Rounds per minute: 600-660 

• Country of origin: USSR 

• Number Produced: 1000000 

PK 7.62 mm light machine gun 

• Max effective range: 850 m 

• Weight loaded: 8 kg 

• Rounds per minute: 650 800 

• Country of origin: USSR 

• Number Produced: 1000000 

Rheinmetall MG 3 7.62 mm LMG 

• Max effective range: 1200 m 

• Weight loaded: 12 kg 

• Rounds per minute: 1000 1300 

• Country of origin: Germany 

• Number Produced: 1000000 

NSV 12.7 mm HMG ’’Utjos” 

• Max effective range: 2000 m 

• Weight loaded: 40 -i- kg 

• Rounds per minute: 700 800 
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• Country of origin: USSR 

• Number Produced: ? 

QJZ 89 12.7 mm HMG light 

• Max effective range: 1500 m 

• Weight loaded: 28 kg 

• Rounds per minute: 540 660 

• Country of origin: China 

• Number Produced: ? 

AK 47 7.62 mm Asault Rifle 

• Max effective range: 250 - 300 m 

• Weight loaded: 4 kg 

• Rounds per minute: 600 +- 50 

• Country of origin: USSR 

• Number Produced: 30000000-50000000 

HK G3 7.62 mm Assault Rifle 

• Max effective range: 400 m 

• Weight loaded: 5 kg 

• Rounds per minute: 600-660 

• Country of origin: Germany 

• Number Produced: 9000000 
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Included Frigate Classes 


This section covers the technical details, as far as they are known, of the frigates that 
were evaluated when determining the input data for the models run. The frigates are listed by 
name of class. If the same class has different details for individual nations, or if the same class 
has been significantly updated over the course of time, more than one setup is included in this 
section. 


F122 “Bremen” 

• Country : Germany 

• Max Speed: 30 

• Economic Speed : 21 

• Tonnage : 3680 tons 

• Lenght:131m 

• Width : 14.5 m 

• Draft: 5 m 

• Main Guns : 1 x OTO Melara Compact/62 

• Aux Guns : 2 X 20 mm Rheinmetall 

• Small Guns : 2 x 7.62 mm MG 

• Missiles : 8 x Harpoon Bk 1C 

• CIWS Missile : 0 

• CIWS Gun : 0 

• AAW Missile : 16 Sea Sparrow RIM-7P 

• Nav Radar : 1 x Raytheon 

• Air/surf radar : 1 ea DA08 / WM25 

• Fire control: lx WM25 / lx Signaal STIR 
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F122 “Bremen” Mod 

• Country : Germany 

• Max Speed: 30 

• Economic Speed : 21 

• Tonnage : 3710 tons 

• Lenght:131m 

• Width : 14.5 m 

• Draft: 5 m 

• Main Guns : 1 x OTO Breda Rapid/62 

• Aux Guns : 2 X 27 mm Mauser 

• Small Guns : 3 x .50 Browning w/ digi sight 

• Missiles : 8 x Harpoon Bk IG 

• CIWS Missile : 42 x Rim 114 RAM Blk I HAS mode 

• CIWS Gun : 0 

• AAW Missile : 16 Sea Sparrow RIM-7P 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea TRS3D / WM25 

• Fire control: lx WM25 / lx Signaal STIR / Optronic 

TYPE 22 “BROADSWORD” Batch II 

• Country : United Kingdom 

• Max Speed: 30 

• Economic Speed : 22 
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• Tonnage : 4800 tons 

• Lenght: 146.5 m 

• Width : 14.8 m 

• Draft: 6.4 m 

• Main Guns : 1 x Vickers 4.5 In MK 8 

• Aux Guns : 4 x Oerlikon GCM/A03 30mm 

• Small Guns : 2 x Oerlikon Gam/BOl 20mm 

• Missiles : 4 x MM3 8 

• CIWS Missile : 32 x Sea Wolf Mod 3 

• CIWS Gun : 0 

• AAW Missile : 0 

• Nav Radar : 1 ea 

• Air/surf radar : 1 x Marconi 968 

• Fire control: 2 x Marconi Type 911 

TYPE 22 “Broadsword” Batch III 

• Country : United Kingdom 

• Max Speed: 30 

• Economic Speed : 22 

• Tonnage : 4900 tons 

• Lenght: 149 m 

• Width : 14.8 m 

• Draft: 6.4 m 
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• Main Guns : 1 x Vickers 4.5 In MK 8 Mod 1 

• Aux Guns : 2 x Oerlikon Gam/BOl 20mm 

• Small Guns : 2 x M 323 Mk44 7.62 mm Minigun 

• Missiles : 8 x Harpoon Bk 1C 

• CIWS Missile : 32 x Sea Wolf Mod 3 

• CIWS Gun : 1 SGE-30 B Goalkeeper 

• AAW Missile : 0 

• Nav Radar : 1 ea 

• Air/surf radar : 1 x Mareoni 967 / 968 

• Fire eontrol: 2 x Mareoni Type 911 

TYPE 23 “Duke” 

• Country : United Kingdom 

• Max Speed: 28 

• Economie Speed : 18 

• Tonnage : 4200 tons 

• Eenght: 133 m 

• Width : 16 m 

• Draft: 5.5 m 

• Main Guns : 1 x Viekers 4.5 In MK 8 Mod 1 

• Aux Guns : 2 x DES/MSI DS 30B 30mm 

• Small Guns : 2 x M 323 Mk44 7.62 mm Minigun 

• Missiles : 8 x Harpoon Bk 1C 
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• CIWS Missile : 32 x Sea Wolf VLS Mod 1 Blk I 


• CIWS Gun : 0 

• AAW Missile : 0 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea Plessey Type 996(1) / Deca Type 1008 

• Fire control: 2 x Marconi Type 911 

“Elli” (Kortenaer Mod) 

• Country : Greece 

• Max Speed: 30 

• Economic Speed : 21 

• Tonnage : 3630 tons 

• Lenght: 130.5 m 

• Width : 14.6 m 

• Draft: 6.2 m 

• Main Guns : 2 x OTO Melara Compact/62 

• Aux Guns : 2 x Oelikon 20mm 

• Small Guns: 0 

• Missiles : 8 x Harpoon Bk 1C 

• CIWS Missile : 0 

• CIWS Gun : 2 x MK-I5 Bk I Phalanx 

• AAW Missile : 24 x Sea Sparrow RIM-7P 

• Nav Radar : 2 ea 
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• Air/surf radar : 1 ea WM 25/ ZW06 


• Fire control: 1 ea WM 25 / Signaal STIR 

“Karel Doormann” 

• Country : Netherlands 

• Max Speed: 30 

• Economic Speed : ? 

• Tonnage : 3320 tons 

• Lenght: 122.5 m 

• Width : 14.5 m 

• Draft: 4.5 m 

• Main Guns : 1 x OTO Melara Compact/62 

• Aux Guns : 2 x Oelikon 20mm 

• Small Guns : 2 x 12.7mm MG’s 

• Missiles : 8 x Harpoon Bk 1C 

• CIWS Missile : 0 

• CIWS Gun : I x SGE-30 Goalkeeper 

• AAW Missile : 16 x Sea Sparrow RIM-7P 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea Signaal SMART/ EW-08 

• Eire control: 2 x Signaal STIR 
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“Kortenaer” 

• Country : Netherlands 

• Max Speed: 30 

• Economic Speed : ? 

• Tonnage : 3630 tons 

• Lenght: 130.5 m 

• Width : 14.6 m 

• Draft: 6.2 m 

• Main Guns : 1 x OTO Melara Compact/62 

• Aux Guns : 2 x Oelikon 20mm 

• Small Guns : 0 

• Missiles : 8 x Harpoon Bk 1C 

• CIWS Missile : 0 

• CIWS Gun : 1 x SGE-30 Goalkeeper 

• AAW Missile : 24 x Sea Sparrow RIM-7P 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea WM 25/ ZW06 

• Eire control: 1 ea WM 25 / Signaal STIR 

“Lupo” 

• Country : Italy 

• Max Speed: 35 

• Economic Speed : ? 
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• Tonnage : 2525 tons 

• Lenght: 114 m 

• Width : 11.3 m 

• Draft: 3.7 m 

• Main Guns : 1 OTO Merlara 127/54 

• Aux Guns : 4 x Breda 40mm / 70 twin compact 

• Small Guns : 2 x Oerlikon 20mm 

• Missiles : 16 x OTO Melara Teseo Mk 2 

• CIWS Missile : 0 

• CIWS Gun : 0 

• AAW Missile : 16 x Sea Sparrow RIM-7P 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea Selenia SPS-774/ SPQ-2F 

• Fire control: 2 x SPG 70/ 2 x SPG 74 / 1 x MK 95 Mod 1 

“Maestrale” 

• Country : Italy 

• Max Speed: 32 

• Economic Speed : 21 

• Tonnage : 3200 tons 

• Lenght: 122.7 m 

• Width : 12.9 m 

• Draft: 4.6 m 
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• Main Guns : 1 OTO Merlara 127/54 

• Aux Guns : 4 x Breda 40mm / 70 twin compact 

• Small Guns : 2 x Oerlikon 20mm 

• Missiles : 4 x OTO Melara Teseo Mk 2 

• CIWS Missile : 0 

• CIWS Gun : 0 

• AAW Missile : 16 x Selenia Albatros Aspide 

• Nav Radar : 2 ea 

• Air/surf radar : 1 ea Selenia SPS 774 / SPS 702 

• Fire control: 1 x SPG 75 / 2 x SPG 74 

“Niels Juel” 

• Country : Denmark 

• Max Speed: 28 

• Economic Speed : 18 

• Tonnage: 1320 tons 

• Lenght: 84 m 

• Width : 10.3 m 

• Draft: 3.1 m 

• Main Guns : 1 x OTO Melara Compact/62 

• Aux Guns : 0 

• Small Guns : 4 x 12.7 mm MG’s 

• Missiles : 8 x Harpoon Bk 1C 
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• CIWS Missile : 4 x Stinger AAM 

• CIWS Gun : 0 

• AAW Missile : 12 x SeaSparrow RIM 7M, Mk 48 Mod 3 

• Nav Radar : I ea 

• Air/surf radar : I x DASA TRS-3D 

• Fire control: 2 x MK 95 / Philipps 9LV 200 MK 1 

“Oliver Hazard Perry” 

• Country : United States, Turkey, Spain 

• Max Speed: 29 

• Economic Speed : ? 

• Tonnage : 4100 tons 

• Lenght: 138 m 

• Width : 13.7 m 

• Draft: 7.5 m 

• Main Guns : 1 x OTO Melara Compact/62 

• Aux Guns : 2 x Boing 25mm MK 38 

• Small Guns : 4 x 12.7 mm MG’s 

• Missiles : 4 x Harpoon Bk 1C 

• CIWS Missile : 0 

• CIWS Gun : I X MK 15 Bk IB Phalanx 

• AAW Missile : 36 x SM-IMR 

• Nav Radar : 1 ea 
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• Air/surf radar : 1 ea SPS-49V4 / Cardion SPS-55 


• Fire control: 1 x Signaal WM28 

“Oliver Hazard Perry” Mod 

• Country : United States 

• Max Speed: 29 

• Economic Speed : ? 

• Tonnage : 4065 tons 

• Lenght: 138 m 

• Width : 13.7 m 

• Draft: 7.5 m 

• Main Guns : 1 x OTO Melara Compact/62 

• Aux Guns : 2 x Boing 25mm MK 38 

• Small Guns : 4 x 12.7 mm MG’s 

• Missiles : 0 

• CIWS Missile : 0 

• CIWS Gun : 1 X MK 15 Bk IB Phalanx 

• AAW Missile : 0 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea SPS-49V4 / Cardion SPS-55 

• Fire control: 1 x Signaal WM28 


115 




MEKO 200 PN “Vasco Da Gama” 


• Country : Portugal 

• Max Speed: 32 

• Economic Speed : 21 

• Tonnage : 3300 tons 

• Lenght: 115.9 m 

• Width : 14.9 m 

• Draft: 6.1 m 

• Main Guns : 1 x Creusot-Loire lOOmm/55 Mod 68 CADAM 

• Aux Guns : 2 x Oerlikon 20mm (turret) 

• Small Guns : 2 x Oerlikon 20 mm 

• Missiles : 8 x Harpoon Bk 1C 

• CIWS Missile : 0 

• CIWS Gun : 1 X MK 15 Bk IB Phalanx 

• AAW Missile : 16 x SeaSparrow RIM 7M, MK 29 

• Nav Radar : I ea 

• Air/surf radar : I x Signaal WM 25 

• Fire control: 2 x Raytheon SPG-51C 

MEKO 200 TN “Yavuz” 

• Country : Turkey 

• Max Speed: 27 

• Economic Speed : 15 
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• Tonnage : 3030 tons 

• Lenght: 110.5 m 

• Width : 13.3 m 

• Draft: 4 m 

• Main Guns : 1 x Oto Breda 127/54 

• Aux Guns : 0 

• Small Guns : 2 x 12.7 mm MG’s 

• Missiles : 8 x Harpoon Bk IG 

• CIWS Missile : 0 

• CIWS Gun : 3 x Oerlikon Contraves Sea Zenith 

• AAW Missile : 16 x SeaSparrow Rim 7P, MK 29 

• Nav Radar : 1 ea 

• Air/surf radar : 1 ea AWS6 / DA-08 

• Fire eontrol: 1 x Signaal STIR 124 
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Code 


This section includes the code used to interweave the NOLH and Hadamard varied de¬ 
signs. It is written in JAVA, version 6.2. 

package extendcvs; 

import java.io.BufferedReader; 
import java.io.BufferedWriter; 
import java.util.ArrayList; 
import java.io.FileReader; 
import java.io.FileWriter; 
import java.io.lOException; 
import java.util.logging.Level; 
import java.util.logging.Logger; 

/ * * 

* 

* @author heiko abel 
*/ 

public class Main { 

public static void main(String[] args) { 

System.out.printin(args [0]); 
readFile(args [ 0], args[l], args[2]); 

} 

public static void readFile(String filenameOne, String 
filenameTwo, String outfile) { 

BufferedReader in = null; 

ArrayList<String> fileOneData = new ArrayList<String>(); 
BufferedReader inSAFC = null; 
try { 

in = new BufferedReader(new FileReader(filenameOne)); 
inSAFC = new BufferedReader 
(new FileReader(filenameTwo)) ; 

String nextlineone = in.readLine(); 
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string nextlineTwo = inSAFC.readLine() ; 


BufferedWriter out = new BufferedWriter(new 
FileWriter(outfile)); 

StringBuffer buf = new StringBuffer(); 

//out.write("cpi_number,responseXn") ; 

buf.append(nextlineone) ; 

buf.append 

buf.append(nextlineTwo) ; 

buf.append("\n") ; 

while ((nextlineTwo = inSAFC.readLine())!= null) { 
fileOneData.add(nextlineTwo) ; 

} 

while ( (nextlineone = in.readLine()) != null) { 

for (String s : fileOneData){ 
buf.append(nextlineone) ; 
buf.append(","); 
buf.append(s) ; 
buf.append("\n"); 

} 

} 


in.close(); 

inSAFC.close(); 

out .write (buf .toStringO ) ; 

out.close(); 

} catch (lOException ex) { 

Logger.getLogger(Main.class.getName()) 
.log(Level.SEVERE, null, ex); 
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} finally { 
try { 

in.close() ; 

} catch (lOException ex) { 

Logger.getLogger(Main.class.getName()) 
.log(Level.SEVERE, null, ex); 

} 

} 
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Event Graph 



Figure 1: Event Graph as developed for the SimKit implementation visualizing the basic concept of the frigate - 
SAFC encounter. 


123 

















THIS PAGE INTENTIONALLY LEET BLANK 


124 



Bibliography 


Abbott, B. P. (2008). Littoral Combat Ship (LCS) Mission Packages: Determining the best Mix. 
Master’s thesis, Naval Postgraduate School, Monterey, CA. 

Abel, H., Sanchez, P. J., Kline, J. E., and Tompson, M. (2009). Frigate Defense Effectiveness 
in Small Craft Green Water Engagements. In: Scythe, Proceedings and Bulletin of the Inter¬ 
national Data Farming Community. Issue 6, Workshop 18. Article retrieved July 01, 2009, 
from http://harvest.nps.edu/scythe.html. 

Alberts, D. S. (2005). Code of Best Practice for Experimentation. DoD Command and Control 
Research Programme: Center of Advanced Concepts and Technology, Washington, DC, 3rd 
edition. 

Ang Keng-Ern, J. (2006). Extending Orthogonal And Nearly Orthogonal Latin Hypercube 
Designs Eor Computer Simulation And Experimentation. Master’s thesis. Naval Postgraduate 
School, Monterey, CA. 

Arabian Business (2007). Heavily armed pirates spark regional shipping alert. Article retrieved 
Nov 28, 2008, from http://www.arabianbusiness.com/506828-heavily-armed-pirates-spark- 
regional-shipping-alert. 

Associated Press (2008). 5 Armed Somali Pirates in a Skiff. Picture retrieved Jan 

28, 2009, from http://www.smh.com.au/news/world/navy-chief-wants-ships-to-hit-back-at- 
pirates/2008/10/09/1223145542078.html. 

Associated Press (2009). 5 Somali Pirates Drown With Ransom Money. Article retrieved Jan 
26, 2009, from http://www.msnbc.msn.com/id/28589921/. 

Baldwin, R. B. (1999). They never knew what hit them : the story of the best kept secret of 
World War IL Reynier Press, Naples, Florida, 1st edition. 

Banks, J., Carson, J. S. L, and Nelson, B. E. (2006). Discrete-Event System Simulation. Prentice 
Hall International Series, Upper Saddle River, NJ, 2nd edition. 

Besheer, M. (2006). US Navy Ships return Fire on Suspected Pirates off Somali Coast. Article 
retrieved Feb 11, 2009, from http://www.voanews.eom/english/archive/2006-03/2006-03- 
18-voa32.cfm?CFID=113496142&CFTOKEN=45001385&jsessionid=883042b3fbbca38 
ccc044c305676797b503d. 


125 




Bitinas, E., Henscheid, Z., and Middleton, D. (2005). Pythagoras: An Agent-Based Environ¬ 
ment. In MORS Journal Abstracts. Military Operations Research Society. 

Bryant, J. and Thompson, S. (2001). Fundamentals of Media Effects. McGraw-Hill Publishing 
Company, Eondon, 1st edition. 

Buss, A. (2009). Simkit. Article retrieved April 6, 2009, from http://diana.nps.edu/Sinikit/. 

Cameron, P. J. (2006). Hadamard matrices. In The Encyclopaedia of Design Theory. O’Reilly 
Asociates. 

Cioppa, T. M. (2002). Efficient Nearly Orthogonal and Space-filling Experimental Designs 
for High-dimensional Complex Models. Dissertation, Naval Postgraduate School, Monterey, 
CA. 

Cioppa, T. M., Eucas, T. W., and Sanchez, S. M. (2002). Exploring The World Of Agent- 
Based Simulations: Simple Models, Complex Analyses. In Proceedings of the 2002 Winter 
Simulation Conference. 

Cioppa, T. M., Eucas, T. W., and Sanchez, S. M. (2004). Military Applications of Agent Based 
Simulations. In Proceedings of the 2004 Winter Simulation Conference. 

Clausewitz, C. v. and Schlieffen, E. G. v. (1905). Vom Kriege. Eerdinand Duemmler Verlags- 
buchhandlung, Berlin, 5th edition. 

Defense Industry Daily (2008). Germany’s E125 Special Eorces and Stabilization Erigates. 
Article retrieved Nov 20, 2008, from http://www.defenseindustrydaily.com/Germanys-E125- 
Special-Eorces-and-Stabilization-Erigates-05119/. 

Defense Update (2006). Active Protection Systems for AEVs - Hard Kill Systems. Article re¬ 
trieved Nov 28, 2008, from http://www.defense-update.com/features/du-l-04/Hard-kill.htm. 

Defense Update (2007). Trophy Active Protection System. Article retrieved Nov 28, 2008, 
from http://www.defense-update.eom/products/t/trophy.htm. 

Defense Update (2008). Iron Eist Active Protection System. Article retrieved Nov 28, 2008, 
from http://www.defense-update.eom/products/i/iron-fist.htm. 

Diehl Defense (2009). RAM - Rolling Airframe Missile. Article retrieved April 6, 2009, from 
http://www.diehl-bgt.de/index.php?id=559&E=l. 


126 



DRS-Technologies (2004). 20mm phalanx projectile recorded close to the muzzle of the gun. 
Picture accessed in May, 2004, from http://www.drs.com/products/popups.cfm?ID=33. 

Dunnington, G. W., Gray, J., and Dohse, F.-E. (2004). Carl Friedrich Gauss : Titan of Science. 
Mathematical Association of America, Washington, DC, 9th edition. 

Efimba, M. E. (2003). An Exploratory Analysis of Littoral Combat Ships Ability to Protect 
Expeditionary Strike Groups. Master’s thesis. Naval Postgraduate School, Monterey, CA. 

Fischer, L. (2006). British 4.5”/55 (11.4 cm) Mark 8 Mod 0, 114 mm/55 (4.5”) Mark 8 
Mod 1. Article retrieved Jan 14, 2009, from http://www.navweaps.com/Weapons/WNBR_45- 
55_mk8.htm. 

Fisher, R. A. and Prance, G. T. (1974). Design of Experiments. Hafner Press, New York, NY, 
9th edition. 

Friedman, N. (2008). Italian 76 mm/62 (3”) Compact; 76 mm/62 (3”) SR - United States of 
America 76 mm/62 (3”) Mark 75 , chapter 17, 76 mm Ammunition, pages 272-275. Jane’s 
Information Group, 17th edition. 

Gander, T. J. and Cutshaw, C. Q. (2007). German 27mm MLG 27 Light Naval Gun System 
. Article retrieved Jan 16, 2009, from http://www.navweaps.com/Weapons/WNGER_27mm- 
145_mlg27.htm. 

Gebre-Wold, K. and Masson, 1. (2002). Small Arms at the Horn of Africa: Challenges, Issues 
and Perspectives. In brief volume 23. Bonn International Center for Conversion. 

Gomar, P. et al. (1996). Gunner’s Mate 1 & C, Chp. 9 : SMS guided mis¬ 
siles, aerodynamics, and flight principles. Article retrieved Jan 09, 2009, from 
http ://w ww.globalsecurity.org/military/library/policy/navy/nrtc/14110.htm. 

Goodnight, J. and Sail, J. (2008). IMP 8 interactive statistics user manual. Article retrieved 
May 22, 2009, from http://www.jmp.com/. 

Grant, G. (2006). Tradoc cuts apaches’ role in deep attack. Article retrieved Oct 06, 2008, from 
http://docs.newsbank.com/openurl?ctx_ver=z39.88-2004. 

Groenke, A. S. (2005). CAS, Interdiction, and Attack Helicopters. Master’s thesis. Naval 
Postgraduate School, Monterey, CA. 


127 



Hand, D., Mannila, H., and Smyth, P. (2001). Principles of Data Mining. The MIT Press, 
London, 1st edition. 

Hickernell, F. J. (1998). A generalized discrepancy and quadrature error bound. In Mathematics 
of Computation, volume 67, Number 221. 

Holland, J. H. and Miller, J. H. (1991). Artificial Adaptive Agents in Economic Theory. In 
The American Economic Review: Papers and Proceedings of the Hundred and Third Annual 
Meeting of the American Economic Association, volume 81, No. 2. American Economic 
Association. 

Hooten, E. R., editor (2008). Janes Naval Weapon Systems 2007-2008 Edition. Sentinel House, 
Alexandria, VA, 42nd edition. 

Horne (2009). Scythe: Proceedings and Bulletin of the International Data harming Community. 
Proceedings in Publication on http://harvest.nps.edu/scythe.html. 

Horne, G. E. and Meyer, T. E. (2004). Data harming: Discover¬ 
ing Surprise. In Skythe. Article retrieved May 22, 2009, from 

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=933815. 

Hughes, W. P. (1995). A Salvo Model of Warships in Missile Combat Used to Evaluate Their 
Staying Power. In Bracken, J., Kress, M., and Rosenthal, R., editors. Simulation Modeling. 
John Wiley and Sons. 

Institute for Electrical and Electronics Engineers (2007). Information for Author: IEEE Trans¬ 
actions, Journals and Eetters. 

International Chamber of Commerce (2008). International Maritime Bureau 
Piracy Map 2008. Article retrieved Mar 3, 2009, from http://www.icc- 

ccs.org/index.php?option=comTabrik&view=visualization&controller=visualization 
.googlemap&Itemid=89&phpMyAdmin=P5XY3CeBeymbElbQ8jr4qlxKlJ3. 

Janes Defense Weekly (2009). Jane’s Defense Weekly. Article retrieved May 22, 2009, from 
http://jdw.janes.com/public/jdw/index.shtml. 

Jones, R. D. and Cutshaw, C. Q., editors (2005). Janes Naval Weapon Systems 2005-2006 
Edition. Sentinel House, Alexandria, VA, 31st edition. 


128 



Kass, R. A. (2006). The Logic of Warfighting Experiments: The future of command and control. 
DoD Command and Control Research Programme, Washington, DC, 1st edition. 

Kusiak, A. (2006). Data Farming: Concepts and Methods, volume 6 of Massive Computing, 
page 279304. Springer US, 17th edition. 

Law, A. M. (2007). Simulation Modeling & Analysis. McGraw Hill, Tucson, AZ, 4th edition. 

Lawlor, M. (2005). Data Farming Cultivates New Insights. Article retrieved April 6, 2009, 
from http://www.afcea.org/signal/articles/templates/SIGNAL_Article_Template.asp? arti- 
cleid=975&zoneid= 158. 

Lockheed Martin (2009). AGM-114K Hellfire Mk. 11. Article retrieved Nov 28, 2008, from 
http://www.lockheedmartin.com/products/HellfireIFindex.html. 

Martin, S. G. (2000). Phalanx Block IB CIWS - ready for the fleet. Article retrieved April 6, 
2009, from http://www.dtic.mil/ndia/ammo/martin.pdf. 

McIntosh, G. C., Galligan, D. P, Anderson, M. A., and Lauren, M. K. (2007a). MANA (Map 
Aware Non-Uniform Automata) Version 4 User Manual. DTA Technical Note, 2007/3:1-7. 

McIntosh, G. C., Galligan, D. P, Anderson, M. A., and Lauren, M. K. (2007b). Recent De¬ 
velopments in the MANA Agent-based Model. Skythe: Proceedings and Bulletin of the 
International Data Farming Community, Vol. 1, Workshop 13:12-15. 

Mitroff, 1. 1. and Featheringham, T. R. (1976). Towards a behavioral the¬ 
ory of systemic hypothesis-testing and the error of the third kind. In The¬ 
ory and Decision, volume 7, Number 3. Article retrieved May 22, 2009, from 

http://www.springerlink.com/content/g5326h70hljm7811/fulltext.pdf. 

Mood, A. M. (1946). On Hotellings Weighing Problem. In Annals of Mathematical Statistics, 
volume 17, Number 4. 

NATO Operations (2009). Operation Enduring Freedom and Active Endeavour. Articles re¬ 
trieved Jan 10, 2009, from http://www.nato.inFissues/active_endeavour/index.html. 

Naval Technology (2009). Naval Technology - the website for the defence industries - Navy. 
Article retrieved May 22, 2009, from http://www.naval-technology.com/. 


129 



Navias, M. S. and Hooton, E. (1996). Tanker Wars: The Assault on Merchant Shipping During 
the Iran-Iraq Conflict, 1980-1988. Tauris Academic Studies Publishers, London, 1st edition. 

Navias, M. S. and Hooton, E. (2001). Joint Publication 1-02: Department of Defense Dictio¬ 
nary of Military and Associated Terms. Office of the Secretary of Defense, London, October 
2008 edition. 

Navy.mil (2007). U.S. Navy Ships Conduct EAC/EIAC Exercises. Article retrieved Mar 3, 
2009, from http://www.navy.mil/search/display.asp7story_id=29911. 

New Zealand Defense Eorce (2009). Organisational Support Branch: Defense Technology 
Agency. Article retrieved April 6, 2009, from http://www.nzdf.mil.nz/ corporate/hqnzdf/org- 
support-branch/default.htm. 

News Navy.mil (2009). Maritime Eorce Protection Command to Activate Oct. 1. Article re¬ 
trieved Jan 10, 2009, from http://www.news.navy.mil/search/display.asp7story_id=15309. 

Newton, P. (2008). Heavily armed pirates spark regional shipping alert. Arti¬ 
cle retrieved Nov 28, 2008, from http://www.cnn.eom/2008/WORLD/africa /11/21/pi- 
rates.ransom.payments/index.html. 

NIST (2009). NIST/SEMATECH e-Handbook of Statistical Meth¬ 
ods: Yates Analysis. Article retrieved April 6, 2009, from 

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35i.htm. 

Norman Eriedman (2006). The Naval Institute Guite to World Naval Weapon Systems. Naval 
Institute Press, 5th edition. 

North Atlantic Treaty Organization (2009). Current NATO issues. Articles retrieved Jan 10, 
2009, from http://www.nato.int/issues/index.html. 

Null, R. and Wallace, J. (2003). Gunner’s Mate 1 & C, Chp. 6 : 

Gun Mounts and Projectiles. Article retrieved Jan 14, 2009, from 

http ://w ww.globalsecurity.org/military/library/policy/navy/nrtc/14110_ch6.pdf. 

O’Reilly, T. (2005). Data Panning, not Data Mining. Article retrieved April 6, 2009, from 
http://radar.oreilly.eom/2005/08/data-farming-not-data-mining.html. 

Pike, J. (2007). AGM-114 Hellfire Employment. Article retrieved Nov 28, 2008, from 

http://www.globalsecurity.org/military/systems/munitions/agm-114-employ.htm. 


130 



Sanchez, P. J. (2006). As Simple As Possible, But No Simpler: A Gentle Introduction To 
Simulation Modeling. Proceedings of the 2006 Winter Simulation Conference, 06:12-19. 

Sanchez, S. M. (2005). Nearly Orthogonal Latin Hypercube Design Spreadsheet, 29 factor 
version. Article retrieved April 6, 2009, from http://diana.cs.nps.navy.mil/SeedLab/. 

Saunders, Commodore, S., editor (2008). Janes Fighting Ships 2008-2009 Edition. Sentinel 
House, Alexandria, VA, 112th edition. 

Schruben, L. (1992). SIGMA: Graphical Simulation Modeling. The Scientific Press, South San 
Francisco, 1st edition. 

Sickinger, L. (2006). Effectiveness of Non-Lethal Capabilities in a Maritime Environment. 
Master’s thesis. Naval Postgraduate School, Monterey, CA. 

Stallman, R. (2002). Free as in Freedom: Richard Stallman’s Crusade for Free 

Software. O’Reilly & Associates, Inc. Book retrieved April 8, 2009, from 
http ://oreilly. com/openbook/freedom/. 

Steelman, E. (2005). Eessons Eearned Erom the Development of the U.S. Navy 

5-Inch Ship Self-Defense Projectiles. Article retrieved Mar 24, 2009, from 

http://www.dtic.mil/ndia/2005garm/ thursday/steelman.pdf. 

Taguchi, G. (1995). Quality engineering (Taguchi methods) for the development of electronic 
circuit technology. In IEEE Transactions on Reliability, volume 44, Issue 22. 

Taguchi, G. (2002). Taguchi methods in ESI fabrication process. In IEEE International Work¬ 
shop on Statistical Methodology. 

Till, G., Carmel, S., Hughes, W. P, Pendley, W. T, and Rubel, R. C. (2008). Maritime Strategy. 
Naval War College Review, 61(2):25-78. 

Tremlett, G. (2009). This is london the capital of somali pirates’ secret intelli¬ 
gence operation. The Guardian online. Article retrieved on Sep. 05, 2009 from 
http://www.guardian.co.uk/world/2009/may/ll/somalia-pirates-network. 

U.S. Army Training & Doctrine Command (1976). TRADOC Bulletin 3. Soviet 
RPG-7 Antitank Grenade Eauncher. Tradoc Bulletin retrieved Mar 3, 2009, from 
http://www.fas.org/man/dod-101/sys/land/row/rpg-7.pdf. 


131 



U.S. Defense Imagery (1982). Photo ID VIRIN: DN-SC-87-01124. 

U.S. Department of Defense (2007). A Cooperative Strategy for the 21st Century Seapower. 

U.S. Department of Defense (2008). National Defense Strategy. Printed copy. 

Watkins, J. (2005). Royal Navy Small Calibre Gun Research to De¬ 
feat the Small Boat Threat. Article retrieved Mar 13, 2009, from 

http://www.dtic.mil/ndia/2005garm/wednesday/watkins.pdf. 

Wright, L. (2007). The Looming Tower. Vintage, 1st edition. 

Yost, D. S. and Masala, C. (2005). Strategies: NATO, the United States, and the European 
Union. NATO Defense College Occasional Paper, 5:4-13. 

Zahle, T. U. (1971). Hit Probability for a Chain-Like Series of Shots - A simple Formula. Naval 
Research Logistics Quarterly, Vol. 18(2):283-293. 


132 



Referenced Authors 


Abbott, Benjamin P. 

Abel, Heiko 
Alberts, David S. 
Anderson, Mark A. 

Ang Keng-Ern, Joshua 

Baldwin, Ralph Belknap 
Banks, Jerry 
Besheer, Margaret 
Bitinas, Edd 
Bryant, Jennigs 
Buss, Arnie 

Cameron, Peter J. 

Carmel, Steve 
Carson, John S. II 
Cioppa, Thomas M. 
Clausewitz, Carl von 
Cutshaw, Charles Q. 

Dohse, Eritz-Egbert 
Dunnington, G. Waldo 

Efimba, Motale E. 

Eeatheringham, Tom R. 
Eiseher, Eeo 
Eisher, Ronald A. 
Eriedman, Norman 

Galligan, David P 
Gander, Terry J. 
Gebre-Wold, Kieflemariam 
Gomar, Phillips 
Goodnight, James 


Grant, G. 

Newton, Paula 

Gray, Jeremy 

Norman Eriedman 

Groenke, Andrew S. 

Null, Roger 

Hand, David 

O’Reilly, Tim 

Henseheid, Zoe 

Hickernell, Ered J. 

Holland, John H. 

Hooton, E.R. 

Pendley, William T. 

Pike, John 

Pranee, Ghiuean T 

Horne, Gary E. 

Hughes, Wayne P 

Rubel, Robert C. 

Kass, Riehard A. 

Kline, Jeffrey E. 

Kusiak, Andrew 

Sail, John 

Sanehez, Paul J. 

Sanehez, Susan M. 

Sehlieffen, Exeellenz Graf von 

Eauren, Miehael K. 

Eaw, Averill M. 

Eawlor, Maryann 

Eueas, Thomas W. 

Sehruben, Eee 

Siekinger, E.R. 

Smyth, Padhraie 

Stallman, Riehard 

Steelman, Euke 

Mannila, Heikki 

Martin, Seott G. 

Masala, Carlo 

Masson, Isabelle 

Meintosh, Gregory C. 
Meyer, Ted E. 

Taguehi, Geniehi 

Thompson, Susan 

Till, Geoffrey 

Tompson, Meredith 

Tremlett, Giles 

Middleton, Donna 

Miller, John H. 

Mitroff, Ian I. 

Mood, Alexander M. 

Wallaee, Jerry 

Watkins, Jonathan 

Wright, Eawrenee 

Navias, Martin S. 

Yost, David S. 

Nelson, Barry E. 

Zahle, T. U. 


133 




THIS PAGE INTENTIONALLY LEET BLANK 


134 



Initial Distribution List 


1 Defense Technical Information Center 
Ft. Belvoir, Virginia 

2 Dudly Knox Library 
Naval Postgraduate School 
Monterey, California 

3 New Zealand Defense Technical Office 
Ochunackzcek, New Zealand 


135 




